22.01.23 17:25:58.628 dream.log WARN REQ 156 Aborted by: Failed to connect to <mariadb://user:_@localhost/database>: Error 1040, Too many connections.
22.01.23 17:25:58.628 dream.log WARN Raised at Dream__server__Session.middleware.(fun) in file "src/server/session.ml", line 25, characters 2-86
22.01.23 17:25:58.628 dream.http ERROR REQ 156 Failed to connect to <mariadb://user:_@localhost/database>: Error 1040, Too many connections.
22.01.23 17:25:58.628 dream.http ERROR REQ 156 Raised at Stdlib__map.Make.find in file "map.ml", line 136, characters 10-25
22.01.23 17:25:58.628 dream.http ERROR REQ 156 Called from Logs.Tag.find in file "src/logs.ml", line 154, characters 14-32
I am using the Dream framework and MariaDB. The SQL pool is initialized by Dream.sql_pool ~size:20 db_url just to avoid too much simultaneous connections.
It is an image server (and the thumbnail page is not divided into small pages, then there are many simultaneous connections, but the pool should prevent an heavy use at the database).
let query_thumb = [%rapper
get_opt {sql| SELECT @octets{thumb_image}
FROM images WHERE oid = %int{oid} |sql}]
let do_thumb request =
match Dream.query request "id" with
| Some x ->
let oid = int_of_string x in
Dream.sql request (fun db ->
match%lwt query_thumb ~oid db with
| Ok Some x -> Dream.respond
~headers:["Content-Type","image/jpeg"]
x
| _ -> Dream.html "error")
| _ -> Dream.html "error"
No heavy use simultaneously by an other application.
If this is a small program and easy to make the change I would also be curious as to whether the same issue occurs with PostgreSQL. With Postgres also there is the option of using pgbouncer, a dedicated connection pool server, so the OCaml code doesn’t need to manage the pool.
How do you actually initialize the pool? (I.e. where is the pool created in your stack of Dream handler)
I’m curious as to why I’ve never seen that issue.
Btw, don’t expect a fix for your issue, antron is not contributing to dream at the moment.
If you look at the definition of sql_pool, you’ll see that it is a function that returns a function:
let sql_pool ?size uri =
let pool_cell = ref None in
(* returns an anonymous function *)
fun inner_handler request ->
begin match !pool_cell with
| Some pool ->
(* use existing pool *)
| None ->
(* .. *)
(* create a new pool *)
pool_cell := Some pool
(* .. *)
Each time sql_pool is called it does allocate a new SQL pool and also returns a anonymous function, but each time this returned function is called, no new pool is allocated.
If you configure your entrypoint as suggested in the readme:
let () =
Dream.run
@@ Dream.sql_pool "sqlite3:db.sqlite" (* a single pool_cell ref is allocated here *)
@@ fun request ->
Dream.sql request (fun db ->
(* ... *) |> Dream.html)
Then Dream.sql_pool is only called once, and each request, the anonymous function it creates is called, which doesn’t allocate multiple sql pools.
Your inversion should only change the behaviour of Dream.sql pool if you were calling Dream.sql_pool inside a request handler, which I believe is not the intended usage.
I have tried both (the original version and my « fixed » version). The first can’t handle more than 150 simultaneous request the second does. I do insert sql_pool in the middleware chain as specified. But it uses a key from the session and then depends of the request. Perhaps it is the key point. I will try a simpler approach.
That what I was getting at, sql_pool is meant to be called only once in the setup and should be independent of the requests.
I use sql_pool in the way described above and I’ve never observed a leak, so I was wondering if I didn’t look hard enough or you have a different usage pattern.