Week 44: what's everyone hacking on this week?

Community
1

I thought it might be nice to have a weekly ‘what is everyone doing this week’ thread since it’s a cold winter Monday morning here in Cambridge :slight_smile:

I’m working on brand new shiny multiarch containers images for opam2 that track all of our supported compiler distributions. There is work in progress on:

  • https://hub.docker.com/r/ocaml/opam2-staging that builds opam2 containers
  • from these, we systematically generate ocaml compiler switches that contain every released compiler, so you can do multi-workspace builds easily.
  • and then we generate compiler variants like alpine-3.6-ocaml-4.05.0 that contain the base compiler, and variants like +afl or +flambda or +safe-string.

These images are rebuilt regularly and will form the basis for the nextgen opam CI infrastructure.

The fun bit is that all these images are currently amd64 and arm64, and soon hopefully will be Windows, thanks to packet.net support. Has anyone looked at the latest Window Server 2016 container support to see if Cygwin is supported yet? I’m installing a VM with Windows but haven’t had a chance to investigate it yet.

Also if anyone has any feature requests for the container infrastructure while I’m working on it, do let me know (or reply here). I’m trying to get this all working to test the 4.06.0rc1 at the moment :slight_smile:

11 Likes
2

I’m working on a compiler this morning which is part of my academic research. The front end is built in OCaml, the back end is LLVM.

The language itself is intended to be a fully safe C-like language, but unlike the other 250 attempts at building such things (say, Cyclone), the goal is to have a language which is not only at first glance almost C (to facilitate easy incremental conversion of existing code), but is almost fully ABI and API compatible with C (to facilitate easy incremental conversion of existing code). The details of how I’m doing this are almost certainly beyond the scope of this thread. :slight_smile: (Also, as academic research, the odds that anyone else will ever use this thing are slim, so the details probably matter only to people reading my papers. :frowning: )

Working in OCaml has been quite pleasant and very very productive. I’m a very new OCaml’er, but I’m finding the language almost relaxing to work in. It’s rare that I’ve had this “it’s soothing to use” experience in other languages. (A few others come to mind over the years: a couple of Lisp dialects, and (oddly) Ruby.) I also managed to get up to speed in the language very very fast, it was only a matter of a couple of weeks. I’m still rusty on a lot of edge conditions and not 100% comfortable but it’s working out quite well.

As long as I’m new and still have these things in my head, I might mention what’s been bad. I should emphasize, nearly everything has been good, from the readability of the language, to the ease of asking questions of the OCaml community, to the quality of the available Emacs editing modes (tuareg+merlin is a fantastic help!) But of course, nothing is ever perfect, and a couple of things have been rough.

One major impediment to my work has been poor documentation. Of course, the bad LLVM documentation has set me back far more than the bad OCaml documentation. (The OCaml documentation isn’t nearly as bad as the LLVM docs, which are truly appalling.)

That said, if you compare the library documentation of OCaml with other systems (say, the C library man pages in BSD Unix, the Common Lisp Hyperspec, or the online library documentation for Python or Ruby), it’s a little on the sad side. It desperately needs improvement. If one wants to claim to be a modern, industrial strength language, one needs modern, and even for psychological reasons modern looking, documentation. (The CL hyperspec is even less modern looking of course.)

However!: there are good OCaml books, and the OCaml language manual isn’t too bad. Also, the OCaml community is great at helping people when they ask for assistance, for which I’m very grateful. (In LLVM world, if you ask a basic question, it can go unanswered forever. In the OCaml community, someone will always help you out.) I’m starting to really love the OCaml community, which is almost as good a feature of the system as the language itself.

A second pain point for me has been OCaml’s, er, plethora of overly opaque build systems. A newcomer unfortunately cannot avoid dealing with a build system almost as soon as they start working, because they need to compile their code, so they can’t avoid having to figure out how to build things.

One problem with the OCaml-centric build systems I’ve touched so far is that the documentation has too much of a “just use these magic commands and don’t worry too much about what they mean” flavor. Unfortunately, I’m perhaps too old a programmer to willingly turn off the “how does that work!?” part of my mind, it desperately wants to understand the semantic content of commands I’m using. The documentation for the various build systems is also quite weak.

So as a result, I’ve been limping along with really, really horrid Makefiles I’ve been constructing by hand. They’re terrible, but I understand what every line means, and when I’ve tried working with the other stuff I haven’t had that grasp on what a given line meant.

At some point soon, though, I’m going to have to bite the bullet and try to use something like jbuilder. I must confess I’m not looking forward to this because it felt too much like magic and too little like something I could understand and control, but that might be entirely my own fault for not spending more hours trying to understand it better.

Anyway, looking forward to another productive week of coding in OCaml. So far the experience has been absolutely terrific! :slight_smile:

9 Likes
3

Very very cool, I look forward to seeing your new language as well! Let us know how your jbuilder porting experience goes. I’ve moved around 50 libraries over to it and–while it’s a young system–it has really changed my experience of working with OCaml code. Now’s the time to get feedback into the issue tracker so it can be polished into a nice long-term shine :slight_smile:

1 Like
4

I suspect it isn’t of much interest to those who don’t have a large pile of C code they need to maintain in any case. It has a simple purpose, which is remediating existing large C codebases. The language is designed with slavish C compatibility in mind. The big difference is instead of approximately 200 forms of undefined language behavior, it is intended to have zero.

1 Like
5

I actually started really getting involved with porting OCaml to OpenBSD many years ago just to have the ability to use the CIL static analysis tool with the full /usr/src codebase. Today, I am curious if we can compile up the OpenBSD codebase with the CompCert certified C compiler, since it has added just enough GNU extensions to work with kernel code. Your language sounds like it fits right in this design space and another nice option to set CC to…

6

A nice initiative indeed! I’m sure we’d all love weekly updates on what everyone’s doing :slight_smile:

I’m currently going through POSA3 to understand resource management. Sadly not hacking anything yet, but soon™!

1 Like
7

Not quite. One will have to update code in order to make it compile under my compiler. My language prohibits a huge number of unsafe operations (and provides alternatives). However, my guess (which will either be validated or invalidated in testing) is that in practice it is pretty fast to make such changes, and in any cases easier to do that than it is to try to avoid undefined behavior with hand work, static analysis, run time checks, etc.

Again, though, whether this ever sees the light of day is unknown. But, either way, I’m really enjoying working in OCaml.

2 Likes
8

I hope I can start to working on a new version of Camomile supporting the recent version of Unicode standard… at least start reading the standard … well, at the minimum I will download it and put it in my iPad … :tired_face: (Thinking ongoing Python/MATLAB projects and many other…)

5 Likes
9

Hooray! Better unicode support for OCaml!

BTW, I would suggest that you make the top line of your old sourceforge page bold and large (the one that points people at the new github pages.)

10

I know I should, but I’m always repelled by all advertisements on sourceforge.net

2 Likes
11

Yet more reason to lead people away from sourceforge!

1 Like
12

I am working on a polymorphic Set data type. This is meant to complement Functor based Set in the stdlib. Hopefully I will be able to push it to opam by the end of the week. :slight_smile:

2 Likes
13

I know it’s off topic in this thread, but mentioning Python reminds me. Does anyone work on a deep learning framework based on OCaml? What we need are automatic differentiation and GPGPU code generation. OCaml potentially excels this area.

14

You want to talk to @ryanrhymes at OCaml Labs! He has been working on a comprehensive framework called Owl for over a year, and gave an OCaml Workshop talk about it in Oxford this year. It has automatic differentiation, and he is looking at GPGPU code generation at the moment.

6 Likes
15

I’m not sure of its current state, but SPOC also provides some GPGPU functionality under OCaml.

16

Wow, looks interesting. Maybe usable for small scale experiments already.

1 Like
17

I’m working on running benchmarks on Arm64 quantifying the overheads for enforcing the multicore memory model. Our current memory model for OCaml is quite strong (sane) compared to C++ or Java. The experiments will inform us whether we need to weaken (crazier) the memory model for performance improvements.

8 Likes
18

Regarding cygwin in Windows containers: I believe it is supported, although running interactively with docker run -it doesn’t work properly. On a Windows Pro desktop with Docker installed you can right click on the whale icon and select “Switch to Windows containers…” (this may involve rebooting the machine to install the feature). Then create a Dockerfile like this one: (from https://github.com/moby/moby/issues/32330)

FROM microsoft/windowsservercore
RUN powershell.exe -Command "iwr https://chocolatey.org/install.ps1 -UseBasicParsing | iex"
RUN powershell.exe -Command "choco install cygwin -y"
RUN powershell.exe -Command "refreshenv"
RUN powershell.exe -Command "[Environment]::SetEnvironmentVariable('Path', $env:Path + ';C:\tools\cygwin\bin', [EnvironmentVariableTarget]::Machine)"
ENTRYPOINT ["powershell.exe"]

Then docker build -t cygwin . and then you can run commands in it like this:

docker run cygwin bash -c '<command>'
2 Likes
19

I’m hacking both OCaml and Jbuilder in a bid to build the former with the latter. It obviously requires some new Jbuilder features. At this stage, our interest is to see if Jbuilder can provide an alternate build system for OCaml developers themselves, given that our Makefiles are alas often broken or unstable for parallel make.

12 Likes
20

Woah, that’s far far easier than the last time I tried! Am installing a Windows Server VM at the moment and will try to push some opam2 images built like this shortly :slight_smile: