# Stack overflow during evaluation (looping recursion?) **URL:** \ **Category:** Learning\ **Created:** [April 16, 2023, 3:25am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955 "2023-04-16T03:25:35Z")\ **Posts on this page:** 17\ **Page:** 1
**Author:** ![abhishes](https://avatars.discourse-cdn.com/v4/letter/a/cdc98d/32.png) [@abhishes](https://discuss.ocaml.org/u/abhishes)\ **Post date:** [April 16, 2023, 3:25am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/1 "2023-04-16T03:25:35Z")
I wrote this code. as you can see it does not contain any recursion and 1000000 million is no where close to the max value of int (so as to cause an overflow) but weirdly this code is getting a stack overflow. why? ```auto List.init 1000000 (fun x -> x + 1) |> List.map (fun x -> x |> Lwt.return) |> Lwt.all |> Lwt_main.run;; ``` ---
**Author:** ![Chet\_Murthy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/chet_murthy/32/1501_2.png) [@Chet\_Murthy](https://discuss.ocaml.org/u/Chet_Murthy)\ **Post date:** [April 16, 2023, 3:36am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/2 "2023-04-16T03:36:47Z")
Is List.map not recursive ? [checking, List.init is tailrec] ---
**Author:** ![abhishes](https://avatars.discourse-cdn.com/v4/letter/a/cdc98d/32.png) [@abhishes](https://discuss.ocaml.org/u/abhishes)\ **Post date:** [April 16, 2023, 3:47am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/3 "2023-04-16T03:47:42Z")
> Is List.map not recursive It seems its not tail recursive `List.init 1000000 (fun x -> x + 1) |> List.map( fun x -> x * 2);;` meets the same fate. Stack overflow during evaluation (looping recursion?). But this is terrible. how does one work with relatively large datasets in ocaml? in today’s day and age mapping over a million item list shouldn’t be a big deal. ---
**Author:** ![Chet\_Murthy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/chet_murthy/32/1501_2.png) [@Chet\_Murthy](https://discuss.ocaml.org/u/Chet_Murthy)\ **Post date:** [April 16, 2023, 3:51am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/4 "2023-04-16T03:51:05Z")
1. IIRC, there are versions of List.map in some of the add-on libraries that tail-recurse after some limit 2. List.map is trivial to implement in a tail-recursive fashion: anybody who is doing anything real will have done so. 3. if you’re actually working with large in-memory data-sets you won’t be using lists. [Per Jean-Christophe Filliatre’s wise arguments] You will be using vectors of one sort or another, and they don’t have this problem. 4. the tail-recursive version of List.map is slower than the stack-y version, b/c the stack-y version performs less allocation. So it’s a trade-off, and for many (actually, most) applications, the stack-y version is better, b/c faster. I’m sure there are more points, but these are the ones I remembered off the top of my head. ---
**Author:** ![smolkaj](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/smolkaj/32/266_2.png) [@smolkaj](https://discuss.ocaml.org/u/smolkaj)\ **Post date:** [April 16, 2023, 4:12am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/5 "2023-04-16T04:12:31Z")
Which version of OCaml are you using? This should not happen with recent versions: [OCaml - The “Tail Modulo Constructor” program transformation](https://v2.ocaml.org/manual/tail_mod_cons.html) ---
**Author:** ![abhishes](https://avatars.discourse-cdn.com/v4/letter/a/cdc98d/32.png) [@abhishes](https://discuss.ocaml.org/u/abhishes)\ **Post date:** [April 16, 2023, 4:13am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/6 "2023-04-16T04:13:39Z")
Welcome to utop version 2.9.1 (using OCaml version 4.13.1)! ---
**Author:** ![Chet\_Murthy](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/chet_murthy/32/1501_2.png) [@Chet\_Murthy](https://discuss.ocaml.org/u/Chet_Murthy)\ **Post date:** [April 16, 2023, 5:07am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/7 "2023-04-16T05:07:19Z")
The implementation of map in list.ml in 5.0.0 does not have this attribute. One could of course copy out the code and add the attribute. ---
**Author:** ![octachron](https://avatars.discourse-cdn.com/v4/letter/o/49beb7/32.png) [@octachron](https://discuss.ocaml.org/u/octachron)\ **Post date:** [April 16, 2023, 6:37am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/8 "2023-04-16T06:37:26Z")
@Chet_Murthy For OCaml 4, you forgot: 1. You can increase the stack limit of your operating system. This is (probably) not necessary with the default size of the OCaml stack in OCaml 5.0.0. And 5.1.0 will have the tailrecursive-modulo-cons implementation. But yes, storing a sizeable amount of small objects in a `List` is not ideal, a simple array is already a better choice. ---
**Author:** ![beajeanm](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/beajeanm/32/189_2.png) [@beajeanm](https://discuss.ocaml.org/u/beajeanm)\ **Post date:** [April 16, 2023, 8:43am UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/9 "2023-04-16T08:43:54Z")
> [@octachron](#): > > But yes, storing a sizeable amount of small objects in a `List` is not ideal, a simple array is already a better choice. I would even add, if the goal is to transform that first datastructure, storing it, no matter how, is less than ideal. Something [iter](https://ocaml.org/p/iter/latest) would be a better choice. ---
**Author:** ![nobrowser](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/nobrowser/32/2099_2.png) [@nobrowser](https://discuss.ocaml.org/u/nobrowser)\ **Post date:** [April 16, 2023, 5:25pm UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/10 "2023-04-16T17:25:47Z")
Isn’t this one of the uses for `Seq` ? ---
**Author:** ![abhishes](https://avatars.discourse-cdn.com/v4/letter/a/cdc98d/32.png) [@abhishes](https://discuss.ocaml.org/u/abhishes)\ **Post date:** [April 16, 2023, 5:43pm UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/11 "2023-04-16T17:43:47Z")
> Isn’t this one of the uses for `Seq` ? But `Lwt.all` needs a list right? is there any other way to wait on multiple Lwt.t? ---
**Author:** ![nobrowser](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/nobrowser/32/2099_2.png) [@nobrowser](https://discuss.ocaml.org/u/nobrowser)\ **Post date:** [April 16, 2023, 6:15pm UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/12 "2023-04-16T18:15:28Z")
Oh wait, you actually have that many Lwt’s. I should have looked at the original post. Hmm, not sure, I’m not into async stuff much, but I’d rethink the whole top level tasking approach. How about, say, splitting the work into chunks of size \> 1 and handing those to the Lwt’s ? ---
**Author:** ![shonfeder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/shonfeder/32/424_2.png) [@shonfeder](https://discuss.ocaml.org/u/shonfeder)\ **Post date:** [April 16, 2023, 6:19pm UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/13 "2023-04-16T18:19:41Z")
What kind of thing do you want to do with the promises? Does something like this that demos just printing them help? ```ocaml let seq_to n = let step i = if i = n then Lwt.return_none else Lwt.return_some (i, succ i) in Lwt_seq.unfold_lwt step 0 let print_nums () = Lwt_seq.iter_s (Lwt_io.printlf "%i") (seq_to 1000000) let () = Lwt_main.run (print_nums ()) ``` ---
**Author:** ![shonfeder](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/shonfeder/32/424_2.png) [@shonfeder](https://discuss.ocaml.org/u/shonfeder)\ **Post date:** [April 16, 2023, 6:25pm UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/14 "2023-04-16T18:25:36Z")
I suppose not because it’s still just computing the stream in sequence… sorry for the misdirect. ---
**Author:** ![beajeanm](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/beajeanm/32/189_2.png) [@beajeanm](https://discuss.ocaml.org/u/beajeanm)\ **Post date:** [April 17, 2023, 1:14pm UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/15 "2023-04-17T13:14:21Z")
If at the end of the day, you need a big list, then yes, you’ll have to pay the cost for it. What Seq/Iter will give you is to avoid creating intermediary list you don’t need. (but I’ve tried that example, minus the `Lwt.all()` life is too short to wait that long, and with ocaml 5 you can map over a list of 1 million items and create the promises.) As @nobrowser mentionned, a very large collection of short live promises doesn’t sound like a great idea. Chunking could be a solution, maybe we don’t need the full list all at once and something like Lwt\_stream would help. It’s difficult to know without any context. ---
**Author:** ![octachron](https://avatars.discourse-cdn.com/v4/letter/o/49beb7/32.png) [@octachron](https://discuss.ocaml.org/u/octachron)\ **Post date:** [April 17, 2023, 1:42pm UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/16 "2023-04-17T13:42:18Z")
> [@abhishes](#): > > But `Lwt.all` needs a list right? `Lwt.all` is not a primitive function. You can write the array version yourself and it will avoid some roundtrips between arrays and lists because `Lwt.all` implementation uses an intermediary array. ---
**Author:** ![jbeckford](https://sea2.discourse-cdn.com/flex020/user_avatar/discuss.ocaml.org/jbeckford/32/3027_2.png) [@jbeckford](https://discuss.ocaml.org/u/jbeckford)\ **Post date:** [April 17, 2023, 5:23pm UTC](https://discuss.ocaml.org/t/stack-overflow-during-evaluation-looping-recursion/11955/17 "2023-04-17T17:23:02Z")
This thread seems related to [Tail recursive, efficient ways to dispatch ~1M requests via Lwt](https://discuss.ocaml.org/t/tail-recursive-efficient-ways-to-dispatch-1m-requests-via-lwt/6318) which was never fully resolved. Very curious to see if rewriting your own `Lwt.all` will work (please post if it does).