Tips on advent-of-code project structure

Learning
1

Hi all,

I’m doing AOC this year in OCaml, and have found that my routine for setting up the daily challenge involves:

  • copying a folder
  • renaming a series of files (because filenames = module names)
  • updating my dayX/dune files to rename symbols
  • updating my dayX/dayXbin.ml file to have the right X value

I.e.:

$ tree
├── day1
│   ├── day1bin.ml
│   ├── day1lib.ml
│   ├── day1test.ml
│   ├── dune
│   └──  input.txt
...
├── day5
│   ├── day5bin.ml
│   ├── day5lib.ml
│   ├── day5test.ml
│   ├── dune
│   ├── index.md
│   └── input.txt
├── dune
├── dune-project
├── fixture.ml
└── readme.md

I’d prefer if all of my daily challenges had the same module basenames and libs names, just living in different child directories. Im 100% sure there is a better way to do this than what I’m doing now. I’d like to just cp -r dayX dayY and be done with it. Each day I’m fine having Lib be my module name, where I can import Lib into my bin file and my test file.

Here’s my repo: https://github.com/cdaringe/aoc-2020

Have some hot :fire: tips? If you have time to burn and want to critique my code, oh boy I’d love that too :slight_smile:

related, but not sufficiently relevant:

1 Like
2

Maybe dune init is what you are looking for?

3

Thx for the response. I realized that I didn’t articulate my target state very well. Ive applied an edit to the above post. Really, what I’m after, is each day having the exact same file name structure, and each being able to use a common fixture module from the root of the project. I’m not sure dune init plays a role in making that now clarified objective possible.

4

Oh I see. You want a project structure like:

day01/
  bin.ml
  dune
  lib.ml
day02/
  bin.ml
  dune
  lib.ml
...

Instead of:

day01/
  day01.ml
  dune
  lib.ml
day02/
  day02.ml
  dune
  lib.ml
...

I don’t think dune works like that, but maybe someone else can chime in. In any case, repetitive daily projects like this is probably not the use case dune is trying to optimize for. Personally I think the latter project structure (above) is not too bad. It should be a directory copy & rename, followed by one file rename at most.

Edit: also the dune file edit, true.

5

Is using a bash script/Makefile acceptable?

6

For new projects, dune is the recommended build tool.

7

That is true. Just it seems that this is much simpler to handle in more general scripting than in dune, since you need access to non-static information (day number).

EDIT: I still think what you want is outside of what dune was intended for (as mentioned by other people above), and rolling dune files and folders via script would be the most straightforward approach.

8

If the dune file declares a different name/public_name for the binary and library in each folder, that should work fine. You still have to change the dune files but that is about it. Otherwise you would need to make each folder a separate dune project.

9

I use the same filenames everyday, and there’s no issue.
I don’t use public_name so that may be a factor.

Typically I have

XX
├── input.txt
├── ocaml
│   ├── dune
│   ├── mainA.ml
│   └── mainB.ml
├── resultA.txt
└── resultB.txt

where XX is the day, input.txt from the AOC website, resultY.txt created manually for posterity.
dune is always

(executable
 (name mainA)
 (libraries utils)
 (modes byte exe)
 (modules MainA))

(executable
 (name mainB)
 (libraries utils)
 (modes byte exe)
 (modules MainB))

(I have some utils library in another folder, shared across days)
mainA and mainB contain something like

let input = 
  let input = open_in "../input.txt" in
  let res = input_line input in
  close_in input;
  res

(* do stuff *)

let () = Printf.printf "%s\n" (whatever result I got)

Workflow is

  • cp -r $yesterday $today
  • rm $today/*.txt; create fresh $today/input.txt
  • cd $today/ocaml
  • edit mainA.ml, run with dune exec -- ./mainA.ml # possibly extra args, eg -debug
  • eventually get a solution, paste into the website, also paste into $today/resultA.txt
  • cp mainA.ml mainB.ml
  • edit mainB.ml until solution, paste into website and $today/resultB.txt
  • `git add $today; git commit; git push

Typical result https://github.com/SkySkimmer/AdventOfCode/blob/master/2019/25/ocaml/mainA.ml

I don’t have an equivalent to day1lib/day1test, everything specific to the day is in mainA/mainB, copied if used in both. Last year there were the “intcode” recurring problems for which I created a separate library.

2 Likes
10

Money. I will do something similar. I think my biggest obstacle was having dune file in the root, which seemed to want to include all other subdirs in compilation. Even if compiling in a dayXX subdir, the root project would assess all other sub dune projects and complain about bin, lib, and test collisions. You made utils a standalone lib, in a non-rooty folder. I was hoping to find a way to just have my utils live in root with a dune file, but i dont think that works. I tried using data-only-dirs, which of course failed.

Thanks for the ref to your project. Man, Scanf.sscanf! Who knew!

11

You can use a rule in dune to copy the file over during build if you don’t mind hacking.

(rule
 (targets utils.ml)
 (deps    ../utils.ml)
 (action  (copy %{deps} %{targets}))
)
12

Using a relative file seems to not take:

File "dune", line 1, characters 0-88:
1 | (rule
2 |  (targets fixture.ml)
3 |  (deps ../fixture.ml)
4 |  (action  (copy %{deps} %{targets}))
5 | )
Error: File unavailable: _build/fixture.ml

Replacing ../fixture.ml with an absolute path /Users/cdaringe/path/to/aoc-2020/fixture.ml works, though is undesirable for obvious reasons. Even though my ../fixture.ml 100% exists relative to my dayXX/dune file, it seems the path is evaluated from a dir inside of the _build dir, instead of the working dir from which the process is executed or the dirname of the dune file.

13

Even though my ../fixture.ml 100% exists relative to my dayXX/dune file, it seems the path is evaluated from a dir inside of the _build dir, instead of the working dir from which the process is executed or the dirname of the dune file.

Hm…it seems to work on my side, let me try it on your actual repo real quick…

14

Hm…I can’t seem to replicate your File unavailable issue on my fork: https://github.com/darrenldl/aoc-2020/tree/test-dune

EDIT: possibly silly idea - do you happen to be building on something other than Linux? (Judging from your absolute path). In which case dune might being doing things differently I’m guessing(?

15

Ya, ok, so the trick is… if you remove the <root>/dune file, the problem should surface. Somewhat confusingly, at the moment in time you forked, the root dune existed just to build fixture, which was also the point of the rule. Thus, rm dune dune-project from root should create a repro. Edit: building on osx :slight_smile:

16

I’m currently using a similar style of project structure and just copying it over each day. That means that in each subdirectory for each day there is:

day01
├── dune-project
├── dune
├── .ocamlformat
├── lib.ml
├── bin.ml
day02
├── ...

This nicely means that each subdirectory is in some sense its own separate project which feels about right from a philosophical direction. Unfortunately this means that any utilities written end up accumulating over the days in the lib.ml file but otherwise is fine :slight_smile:.

17

I’m using a template directory which I copy into a new day’s directory each time. Having the template as a separate directory means I can update and test the template itself.

I also have a script at the root of my repo that automatically fetches my input (I saved my cookies in a gitignored file in the repo) and copies over the template. It renames my dune library to the day number, but the file structure is the same each day.

(library
 (name day_7_solution)
 (libraries async bignum core expect_test_helpers re topological_sort)
 (inline_tests
  (deps input.txt))
 (preprocess
  (pps ppx_jane)))

day_7
├── dune
├── import.ml
├── input.txt
├── main.ml
└── main.mli

I haven’t gotten to the point yet this year, but when I will inevitably write some shared library between solutions, I’ll put it in the root of the project and create a new dune library that I can just depend on in each day’s solution dune file, rather than trying to copy over individual files. That lets dune think about the dependencies instead of me trying to write them out :slight_smile:

1 Like
18

I did exactly this, working well :slight_smile: My repo: GitHub - yawaramin/aoc2020: Advent of Code 2020

19

You may want to use drom (https://github.com/ocamlpro/drom) at some point: it has a notion of skeleton, and is able to perform substitutions on these skeletons to instantiate them for every new project.
If you look at the master branch, it currently has skeletons for library, program, mini-lib (same as library without docs, tests, etc.), mini-prg and now ppx_rewriter. The substitution language is far from perfect, but is being improved for every new skeleton.

With drom, you can define your own skeletons in ~/.config/drom/skeletons/, by mimicing the same structure as the share/drom folder in the sources.

1 Like
20

I’ve kept things simple. Instead of separate projects for each day, I just have one library project. Each day i add a new module day_one, day_two etc, and I run tests using https://github.com/janestreet/ppx_expect that run the two problems for each day. I’ve found this to be a pretty nice setup, as I can still keep some shared code in modules that are used by each day’s specific task.

1 Like