OpenAI ChatGPT writing OCaml

Community
21

Nice! Thank you for sharing!

Best regards

22

I will read that as ā€œWhat is your thought on ChatBot and creating software code?ā€

As one who has decades of experience creating code and having worked with Prolog and understanding how hard creating NLP chatbots have been to create, this is impressive. But then I also know that there are Achillesā€™ heel that are often not known and/or disclosed.

I spent the last few days trying out some Prolog questions of ChatBot to see the responses. I did not grade or test each answer and only commented on a few but added this overall comment:

For those new to Prolog, be aware that many of the answers below may contain flaws or inaccuracies. It is advisable to critically evaluate each answer and spend time noting both the strengths and weaknesses. Trust but verify.

Clearly as noted in some of the replies above, ChatBot can only do low level of math, and even then simple things like addition of numbers several digits long can be wrong.

So only just a few minutes ago I came to this site to see if similar OCaml questions were asked of ChatBot and found this topic. It is nice to see that others are posting factual responses so that down the road we will have milestones to measure against.

I am finding that in some ways ChatBot learns rather quickly. Even asking the same question a day apart returns significantly better answers, e.g. comments in the code and explanations after. But also in understanding how the tokens of an LLM are used and how this reflects back in the responses is helping me to narrow down on those Achillesā€™ heel or mines to avoid in the mine field.

While I use to do OCaml quite a bit it would be interesting to see more questions and answers supplied in this topic as more baselines for future reference. :grinning:

23

Terrifying. To paraphrase Hemmingway: I mistrust bots, especially when their English prose holds together.

1 Like
24

Today in logging into ChatGPT was greeted with

image

Still gets a basic addition wrong. (ref)

1 Like
25

Some might find this illuminating: https://twitter.com/dsmerdon/status/1618816703923912704

Why does chatGPT make up fake academic papers?

By now, we know that the chatbot notoriously invents fake academic references. E.g. its answer to the most cited economics paper is completely made-up (see image).

But why? And how does it make them? A THREAD

26

Thanks.

A few days ago I asked ChatGPT (ref)

What are the important research papers for understanding ChatGPT? Please include links to PDF versions of papers.

some of the links

  • lead directly to a downloadable copy of the paper
  • lead to 404 results but the paper was posted elsewhere and found by searching for the title of the paper
  • some I could not find and thought were behind a paywall or removed
  • some lead to a different (wrong) paper

but never thought they might be a hallucination.

Seems that hallucination will become a common word over the next year as I only see more and more people making use of such technology. The problem is that such hallucinations will be posted and start to make their way into search results.

27

Over on the birdsite, @cwebber coined the term ā€œmansplaining as a serviceā€ to describe ChatGPT. Boy howdy, thatā€™s spot-on!

4 Likes