Synchronous discussion channel for compiler contributors

[giltho]

Hi @gasche!

@gasche

Your post motivated me to restart the compiler-newsletter which may be a way to share knowledge of what is going on.

I am extremely excited about this! Thank you for starting again

@gasche

@Gopiandcode and @giltho, when do you think would be a good time for office hours? Do you have other suggestions to make it easier to contribute?

First, I’d like to say that I am sorry if I have sounded criticizing. I would like to make one thing clear: the main two reason I don’t contribute more to OCaml/the standard library are: 1) lack of knowledge (or the compiler mostly), and 2) more importantly, lack of time. At least for me, it is not the lack of accessibility that have prevented me from participating. All previous issues I have opened on the compiler have been addressed in a timely manner, the only PR I have opened on the library was eventually merged following helpful comments, and the other PR I have open is blocked only by me. Merging a PR takes a lot of time, yes, but not disproportionate compared to the amount of maintainers and their available time.

That being said, I do see a lack of engagement from the community (including myself). How many people actually open PRs on the library? What Lean has, Rust has, and other smaller community have, is an official and synchronous communication medium. Both Lean and Rust have an official Zulip with:

• Help from newcommers
• Open technical discussion from the maintainers
• Channels for discussions contributions

I find the Zulip interface a bit cluttered, but it is great because it is free for open communities, allows access without an account, and allows for efficient discussion management.
The second point is interesting and I had not realised until writing this: the maintainers of Rust discuss together openly on Zulip. Even for planning/triaging meetings, the agenda is discussed in advance in the open (in addition for the meeting to be open). See for example this discussion. This leaves space for people to participate and engage.

My conclusion is that:

1. I don’t feel like the maintainers are putting specific barriers to participating
2. For me the issue is an issue of community engagement
3. It is possible that the maintainers could improve the community engagement with additional effort.

I’d be up for a call to discuss this, though I’m not sure how helpful I can be. I could maybe dedicate time to help setting up something like this?

[octachron]

We could certainly give a try at office hours, for asking questions about the
compiler. With some luck, it could dispel some of the mystic around compiler
design discussions.

However, most of the written discussions on OCaml design already happen on
public online forums. Certainly, there are physical discussion that happens in
Paris (or London or New York) but this is due to the higher density of people
working on the OCaml compiler in this area, and this is not limited to INRIA
halls. Possibly, @gasche and me should make more efforts summarizing some of
those discussions online.

In particular, a point to keep in mind is that most compiler projects are
designed and implemented by one or two persons. In other words, like in many open
source projects, the best way to start being involved is by starting to
contribute.

Certainly my advice is to start with small tasks, like improving the manual
documentation or tweaking an error message. Nevertheless, anyone is welcome to
review the modular explicit pull request. I just don’t have any right as a
maintainer to expect that a new contributor will be familiar enough of the compiler internal to contribute to this review on their first participation.

[giltho]

@octachron

However, most of the written discussions on OCaml design already happen on
public online forums.

What specifically do you refer to? Here or github issues?
In any case I think there’s a lack of synchronous official channel. There’s an ocaml discord, but it’s not quite doing the job of Rust/Lean’s Zulip.
There’s here, but it’s not synchronous, and not quite the same type of post.

[octachron]

I was thinking mostly of the github issues, pull requests, RFCs, discussions, and this forum, and the ICFP ML and OCaml workshops.

The discord channel and libera chat irc is already available for synchronous discussion, and I am available there for discussing compiler internals to anyone interested.

[giltho]

Thanks @yawaramin for splitting this up

@octachron I think there’s value in having a well-organised Zulip channel where informal discussion goes on.

• People can’t really ask questions through pull requests / issues
• This forum is not synchronous
• Discord is not linkable / open without an account. And it is definitely not well-organised, in e.g. the beginner channel, questions intertwine, there is no thread…
• ICFP is not free

I think there is value in copying community models that are clearly working, but I also won’t insist too much if nobody else believes it’s useful.

[zapashcanon]

It reminded me that two years ago I created a Zulip organization for OCaml but then forgot about it (I’m happy to give ownership to any maintainer if someone is interested).

I agree with @giltho. I’m using libera.chat myself but it may not be for everyone. I’m not using Discord/Slack because it is not free software and is much less pleasant to use than Zulip.

Zulip has been working very well for me in many contexts and I’d be happy to see the OCaml community starting using it.

[gasche]

(Providing some extra information/context, specifically in the context of prospective contributions to github/ocaml/ocaml, not in general contributions within the OCaml ecosystem.)

People can’t really ask questions through pull requests / issues

We enabled the Github Discussions, which are fine for asking questions about compiler development. Other community spaces (such as the present forum) are also fine, but my impression is that if the question is specifically about the compiler distribution, it makes sense to do it at the same place where the compiler codebase is stored and evolves. (For example this allows easy cross-linking of issues and PRs, it makes discussions somewhat more searchable, etc.)

This forum is not synchronous

I’m not sure why it would be important to specifically have a synchronous discussion medium. (Personally I tend to avoid them because they are addictive for me and thus too time-consuming, so when I participate to Zulip channels I do it asynchronously. I am happy to let other people pick whatever they prefer.)

Discord is not linkable / open without an account. And it is definitely not well-organised, in e.g. the beginner channel, questions intertwine, there is no thread…

And Discord is a proprietary walled garden, which I think is a big problem. But then I believe the idea is to let people create discussion channels wherever they already are – if everyone these days insists on using Discord to chat, it is kind of losing battle to insist on staying on libera.chat only. (People hoped that Matrix bridges would solve this probelm, but it hasn’t really worked out.)

(The OCaml Discord (or IRC, etc.) are for the whole OCaml community, not specifically compiler development. I think it is fine to have several distinct places for general OCaml discussions.)

ICFP is not free

There are many issues with ICFP, including the fact that it is unreasonably expensive. But I think that the idea is that we have some in-person events as well, and those are important. “FUN OCaml” is one attempt among other to have more sustainable community events (than ACM-colocated workshops, which I suspect are a lost cause for non-mainly-academic events). For another example, I would like to have an OCaml room at FOSDEM (or possibly a room shared with other elegant programming languages, that would be nice, but everyone wants their own room); but this requires work, so I keep hoping that someone (else) will make it happen.

[yawaramin]

My 2c: synchronous communication is fine to do occasionally but on the whole it’s more important to manage async better so that, eg, things don’t slip through the cracks. People are contributing, but sometimes they don’t do it in exactly the right channel and then they get ignored.

Happy to discuss more if there is interest.

[nojb]

Looking at the discussion from a distance, one conclusion that jumps to the eye is that the discussion here is perhaps not as much about substance as one of visibility.

While it is true that most discussion already happens in the open, it is happening in a bunch of different places and in ways that are not visible to the majority of the public. There are various reasons for this:

• various semi-closed forums (caml-devel mailing list, the OCamlLabs Slack, etc) where not much discussion happens, but when it does it is not visible to the public,
• various forums which are not publicized in a consistent manner (caml-devel meetings, triage & other specialized meetings),
• asynchronous discussions that evolve over long timeframes (PR, RFC discussions, also language committee discussions) that give the impression of being stalled to the casual observer, etc.
• discussions that happen in person in various contexts (INRIA, etc)
• most (all) core contributors have a day job which leaves very little time to do outreach with the public on top of everything else

I remember when I started contributing to the compiler that I also was quite puzzled as to where the development of the compiler was actually being discussed. When I eventually got subscribed to the caml-devel mailing list, I was sure I was going to finally access the place where these famous discussions were taking place. Alas, I was sorely disappointed: almost nothing is discussed there, and it is mostly used to synchronize meeting times, etc

The naked truth is that the development of the OCaml compiler is driven forward by a very small number of active developers, who mostly do so on the side of their day job. This looks to me like one key difference with those other languages (Rust, …) which, thanks to the vastly superior means available to them, can afford to hire full-time developers to work on their toolchain, plus support staff, etc.

Having a place to centralize existing discussions (Zulip?) with the possibility of synchronous discussions could help, but it will work only if attracts enough core maintainers, otherwise it will be yet another forum like the many that exist already.

Cheers,
Nicolas

[bluddy]

I think starting in a new location (zulip), despite the tool being excellent, is going to spread resources even thinner.

discord, while having its disadvantages (mostly that of being closed source), has copied some of zulip’s ideas and can now provide a competitive alternative, at least in some ways. We could open a #compiler-dev channel and set it to act like a forum, which gives the advantages of long-form mixed with synchronous short form discussions which zulip provides. We already have this activated in the discord server’s #forum channel, it’s just not very popular. Alternatively, we could simply add a normal, synchronous #compiler-dev channel to the discord.

The advantage of either of these options is that the discord server already has a large following (5000 members, with 500 online as of this moment), and amplifying the same location provides a feedback effect and allows more people to get involved in compiler development.

Between a dedicated forum at the github page and potentially a dedicated synchronous channel at the discord, I think the main needs would be met without adding yet another location. And as @nojb says, these avenues need to actually be used by the core maintainers in order to have value.

[giltho]

@bluddy

Alternatively, we could simply add a normal, synchronous #compiler-dev channel to the discord.

First I don’t think it’s only a matter of compiler-dev. I’m talking about enhancing community engagement, this includes libraries, ecosystem etc. For me the only tool that does this well for the OCaml community is this forum. I feel like nobody will be able to be involved in the compiler before people feel engaged in the community. Note that the original comment that sparked this debate from @Gopiandcode was about participation to the ecosystem, not specifically the compiler.

Second, here’s what I see when I (sometimes) click on the OCaml server in discord

And now the same thing for Rust:

I won’t click on each and every channel in discord, and read all messages until I find something to engage with.

I will, however, go through the list of subjects in the Rust operational semantics team and figure out if I want to engage with anything.

More importantly, the discord server doesn’t feel “official”, but more “community supported”. This is not where discussions happen, it’s been acting more as a helpdesk (from what I’ve been able to see). I feel like, as @nojb said, there’s a lack of centralised and official/advertised discussion place for anything OCaml (including compiler/language development). Which does not preclude these discussions to be summarised or linked to github issues/PRs, or RFCs.

Also, I’m just discovering the github discussions after 9 years of having OCaml as my main language.

That being said, I fully agree with the fact that Rust and Lean have great resources and that makes everything much easier. Re spreading resources thin, I think that creating a Zulip would mean deprecating and eventually removing discord, and other means of discussion. I’d keep:

• github issues/PRs/RFCs
• zulip
• This discuss

That’s it.

[shonfeder]

Personally, I’d be keen to move my own communications around ocaml dev from discord and slack to zulip — as much as possible, anyhow. Let’s give it a go!

[blement]

I think this is a very interesting discussion to have, as OCaml could harvest more contributions from the community. I’ve been doing some contributions to the compiler this year, and a lot of effort has been put in making the repo accessible to contribute (CONTRIBUTING.md, HACKING.adoc, BOOTSTRAP.adoc, typing/HACKING.adoc, etc…). The review process have been very smooth (although I come from Inria so I know most of the people I interact with). That being said, I’ve been on the Lean Zulip for some time, and I’m impressed by the activity there. Besides the friendly help to newcomers, there are technical discussions on contributions to libraries and to the core. Just telling/discussing publicly the general directions of various parts is very helpful for people who want to contribute. Making the remaining tasks explicit (especially if there are low hanging fruits) enables more crowd-sourced contributions.
The channel system of Zulip (although it has some downsides) is quite well adapted to discuss technical topics, sometimes with long delay between messages. It can be done with Discord’s threads, but they get out of focus after a day of inactivity.
Whatever the platform, I’ll be happy to try to contribute more to the community building

[octachron]

@giltho

I feel like nobody will be able to be involved in the compiler before people feel engaged in the community.

This is objectively false: there is a regular flow of new unique contributors to the compiler. On a typical new release of the compiler, there are between 10 and 20 new contributors (counting changelong entry authors). So people are already getting involved with the compiler.

@giltho

More importantly, the discord server doesn’t feel “official”, but more “community supported”. This is not where discussions happen

If discussions do not happen , it is a matter of people participating. Changing the medium will not change this, at least not overnight. I don’t think anyone refrain from discussing because there is not an "OCaml foundation’s approved’ sticker on the server.

@giltho

I think that creating a Zulip would mean deprecating and eventually removing discord, and other means of discussion.

You cannot delete communities because you don’t like them. Even in term of deprecation, communities do not disappear, or shift use patterns that swiftly. For instance, the discord server has not really displaced the irc #ocaml channel on liberachat (which is still active and bridged to discord channel) , it has merely complemented it . Even if a new zulip was successful, I would expect that the discord server keep being active.

Overall, personally I don’t really care about the discussion medium (if there are at least an open source way to participate). Similarly, I believe that technical details of the discussion medium are mostly irrelevant . Some of my fondest memories of online discussions on the OCaml compiler design are from the irc channel (from ten years ago?). I am using the discord server not by personal choice, but merely because this the channel used by most users.

If people want to experiment with a zulip why not. However, you should be aware that this is mostly asking me and the other maintainers active on synchronous discussion channels to split ours attention to one more communication channel which will likely not result in any increase of activities in short term.

[giltho]

@octachron

However, you should be aware that this is mostly asking me and the other maintainers active on synchronous discussion channels to split ours attention to one more communication channel which will likely not result in any increase of activities in short term.

I am not asking for anything, I am merely proposing and arguing for what I believe is the best option. In the end, I am not a maintainer and cannot make this decision. I am willing to spend time helping to centralise the community and advertise for this potential Zulip if it happens, but I can’t force it if people don’t want to, and I certainly cannot force the maintainer. And really, I’m not trying to have a heated debate .
Also, as mentioned before, I am personally not personally unhappy with how the compiler is maintained, I have contributed in the past and what has prevented me from contributing more is mainly time.

@octachron

Even in term of deprecation, communities do not disappear, or shift use patterns that swiftly.

I am aware that it would not happen overnight. It would be a slow process, and would require a lot of advertising and communication. That being said, it would only work if the most active people in the community actually accept to move to Zulip.

Again, this all started from this blog post from Kiran and this response from Gabriel. Gabriel asked “If someone in your position feels unable to approach maintainers to offer contributions, what of newcomers?”. I am proposing what I believe is a step in the good direction, but in the end, i have no control and can only answer with what I know have witness working for other community.

Ultimately, I feel like I have almost engaged more with the Rust community than the OCaml community, even though I barely programmed in Rust, and my feeling is that it’s mainly because of how the community is structured, but I could be wrong.

[gasche]

I think it’s great to have this discussion and be able to take various ideas seriously. Please be understanding if the tone of old-timers gets a bit defensive sometimes (as in any organization when processes are discussed): we have a long experience of having these discussions from time to time, and experience suggests that some of those change proposals lead to good outcome (moving from SVN to github: wise move!), but others end up changing very little, and it is hard to tell in advance which is which.

As @zapashcanon already mentioned, he restarted the Caml Zulip based on this discussion. We can always give it a try and see how it goes. (But I would not suggest trying to push people into moving there; some already have communication channels they are comfortable with.)

[avsm]

Since all the options are being thrown into the mix, I’ll also just point out that Discourse (this forum) also has synchronous chat builtin that we haven’t activated. There’s plenty of scope to just use the existing community here and to restructure it: Discourse Chat Features | Discourse - Civilized Discussion

[zapashcanon]

@kit-ty-kate noticed there is also an ocaml.zulipchat.com (with an o, which is why I had to call the other caml.zulip.chat) but it seems to be invite-only and we don’t know who is the owner. It would be great to have a redirect or change the URL if everyone agree (IIRC it’s not too hard to do in Zulip).

[bluddy]

Oh I totally forgot about that. It was an experiment @perry and I did long before Zulip got any traction (via Rust I guess?). I was considering Zulip for its technical superiority but it just seemed too foreign to people. Since I can’t find any way to change the URL, I’ll gladly hand it over to the community

[giltho]

@gasche

But I would not suggest trying to push people into moving there; some already have communication channels they are comfortable with.

I do understand, though I guess what I’m trying to say is not that Zulip specifically is a superior channel of communication. What I think I’m trying to say is that we should push to reduce fragmentation of the community as much as possible by encouraging one “official” / “recommended” (phrase it how one feels most comfortable with) channel of communication – and I think Zulip is most adapted for that.

The Rust community still has a discord, a discourse etc. But the maintainer and all rust teams discussion officially happen on Zulip (including with some Zulip-specific automation to the workflow, see the --fetch-zulip-id flag for instance). This, in turn, is an incentive to centralise the community there, and for the community to engage with these discussions.

Also, this structure is enforced by the ecosystem leaders (i.e. the Rust core team). Rust’s CONTRIBUTING.md explicitly contains the following sentences:

The best way to get started is by asking for help in the #new members Zulip stream. We have a lot of documentation below on how to get started on your own, but the Zulip stream is the best place to ask for help.

It is recommended to ask for help on the rust-zulip, but any of these platforms are great ways to seek help and even find a mentor!

In addition, Standard Library Contribution Guide, and other core components of the Rust ecosystem, such as Cargo explicity structure their workflow and communication around Zulip, and redirect any newcommer to these channels

If you have a general question about Cargo or its internals, feel free to ask on Zulip.

So yes, people have their preferred channel of communication, and I’m not saying everybody should be moving now to Zulip or that discord should go away. But I do think that the model of “organising the community, language, standard library, compiler etc.. around a single polyvalent (synchronous and asynchronous) communication channel” works.

Note that I keep arguing for Zulip but I do have big issues with the app. Including the fact that switching communities on the MacOS app just doesn’t work… I’m not saying it’s perfect, but it did work for other similar communities.