Hi,
While playing with chatgpt, I tried to see what it could be able to do if I ask it to teach ocaml, so I got this tutorial: decapode314.free.fr/dev/ocaml-en.html
It’s not perfect, but I think it’s quite good, and could be improved.
While willing to make a second attempt, I suddenlly had the idea of asking to integrate the introduction of the language into a short narration: decapode314.free.fr/dev/oca-tale-1.en.html
I found it very fun, so I asked it to create more that 20 short scenes, where an exemple of code is integrated into the story. (not perfect, but I found it interesting, and it could be improved)
I don’t know if other people would be interested, but I just try it. Maybe it would be fun to make a contest, of using chatgpt to create a tutorial, or any other kind of creative use of chatgpt.
I’ve found it to be randomly misleading about even fairly basic stuff, or at least it was several months ago. It seems to do better with python where there are a million examples for it to copy.
It indeed depends on what you expect from it.
Xavier probably doesn’t expect the same from his tools than the everage beginner and hobbyist.
I was just suggesting a contest for fun, to see what funny things we could do with it.
I have experience with GPT and alike and Rust language. So far, it usually gets types wrong most of the time (for anything non-trivial), so I would expect the same problem would happen with OCaml as well.
My tongue-in-cheek comment quoted above wasn’t very clear, so let me elaborate.
When someone posts a question in this forum, they expect, and deserve, and most of the time get a correct answer written by a competent member of this community. If they wanted to have ChatGPT’s answer to their question, they could have asked ChatGPT themselves, no need to post here.
Posting IA-generated answers on this forum is wrong, in my opinion, and should be prohibited. Those answers are often wrong and need to be reviewed and corrected later by humans. The end result is more noise in the discussion and more work for the human experts (it’s more pleasant to write a good reply than to correct IA-generated gibberish). It also hurts the reputation of this forum as the place where very competent OCaml programmers reply to all sorts of OCaml-related questions. Finally, the AI-generated answers get picked by search engines and end up in training data for the next generation of AI systems, which is problematic.
So, have all the fun you want with ChatGPT, but please don’t post ChatGPT-generated answers in this forum.
My anecdotal experience with ChatGPT and OCaml is that last year I asked it how to iterate on the unicode characters of a string, and its answer was plain wrong. I just tried it again, and the result was basically the same. Cases like that make me suspect that it may be worse for beginner questions, since a beginner may not know why its response is incorrect.
I don’t see the harm with using it to generate stories just for fun, but I also suspect there’s not much of an appetite for doing that on an OCaml forum.
Edit: in case anyone reading this wonders what the “correct” answer to my prompt is, see String.get_utf_8_uchar.
I would go a little further than that. If someone cannot answer a question posted on this forum with a reasonable degree of knowledge, whether a beginner’s question or not, they are unlikely to have the knowledge to correct the hallucinations and other errors of AI generated content about the question.
So such uncurated content is decidedly unhelpful, particularly as those who find ChatGPT useful or “fun” can consult it as frequently as they wish without posting the results here.
I didn’t realise that there are people posting copy-pasted answers without even checking if the answer is correct or not.
But in the end this off-topic discuss with 9 messages, also polutes the original message that in the end recieved 0 related answers.
Maybe you could also add in the FAQ something about messages getting drown by a flood of off-topic messages. Usually it is recommended to start a new thread in these kind of situations.
Off topic is fine because if/when we deviate enough from the original thread and admin can split the conversation into a separate conversation.
(And this off topic might be eligible for this treatment)
Thank you, I didn’t know this tool. I’ll try to test it.
I got this demo done with chatgpt.
Honestly this is not something I would be able to do alone with my modest skills.
I think I’ll try to make an html version of this old game, with the code of this demo.
======
Edit: Finally finished to port one level. I’m quite happy with the result.
I think some of the people in this forum may not be familiar with how good LLMs are at writing code. While not perfect, they can remove the massive time overhead of involved in learning new APIs or monotonous tasks, and sometimes can virtually write a whole application for you. Most LLMs aren’t that great at writing OCaml because there are so few instances of code out there, but they are quite excellent at writing (and debugging) Javascript and Python, for example.
Considering chatgpt is able to translate one programming language into another one, the lack of available ocaml code is not that bad.
ChatGPT is also better in English than French for natural language, and it’s quite obvious when we use it a little that chatgpt “thinks” in English, and translates to French before to send you the answer.
It can also “think” in Python, and then translate to OCaml.
I’ve been at Recurse Center for the past number of months where lots of people are frequently using LLMs. I’ve used ChatGPT and mostly Claude, the free versions, and I haven’t tried ChatGPT4o yet. With these qualifications, a few conclusions:
LLMs aren’t just trained on massive amounts of data but also require massive amounts of data to function effectively
I’ve been mostly working in OCaml, Tcl/Tk and R. My experience is that both ChatGPT and Claude fail pretty miserably with all three.
I’m not surprised that they’re not good with OCaml b/c it’s a niche language. The code that’s available online is generally pretty good; there’s just not enough of it.
I am a bit surprised about Tcl/Tk b/c although it’s niche now, it was big previously. But the computing world was smaller then and I suspect the sheer volume of available code isn’t enough.
ChatGPT in particular tries to write Tcl/Tk as Python. I don’t know if this is b/c of the sheer volume of Python code out there and/or because of Tkinter, or something else.
I was surprised about how poor its R code is, given the language’s popularity. I suspect it’s because R is often being used for idiosyncratic purposes.
The more specialized one’s task, the worse that LLMs perform. They’re great at generating Conway’s Game of Life, terrible at implementing domain-specific algorithms.
They’re great at generating boilerplate.
I find them useful for unblocking myself, and would like to get in the habit of using them more for this purpose.
Claude 3 Sonnet seems to be indeed a little bit better, but there are only a limited number of requests that can be done by day, with the free version.
Finally included 3 levels in the mini-game (html5/canvas), from the original game I made in 2008.
And I made a mini-release of the 2d physics code as a reusable mini-lib. Maybe this is the first LLM generated ocaml lib arround.
Now I’m trying to see if I can get a minimalist RTS game done with ChatGPT. I already have a very minimalist version, which works with just a text-console output. Will see what kind of result I can reach with it.
