Haskell: save one listitem at a time to file

Question

I'd like to save a huge list A to a textfile. writeFile seems to only save the list at the very end of the calcultaion of A, which crashes because my memory is insufficient to store the whole list.

I have tried this using

writeFile "test.txt" $ show mylistA

Now I have tried saving the elements of the list, as they are calculated using:

[appendFile "test2.txt" (show x)|x<-mylistA]

But it doesn't work because:

No instance for (Show (IO ())) arising from a use of `print' Possible fix: add an instance declaration for (Show (IO ())) In a stmt of an interactive GHCi command: print it

Can you help me fix this, or give me a solution which saves my huge list A to a text file?

Thank you

There shouldn't be a problem in the first place. I tested with `main = writeFile "test.txt" $ show [x | x — firefrorefiddle, Aug 01 '13 at 10:54
Yeah, you are right, @jozefg Solution works fine, but my memory usage seems to be caused by the generation of mylistA. myListA is basically [link](http://stackoverflow.com/questions/1139953/haskell-faster-summation-of-primes) @ephemient s solution, where i add some additional contraints like `myListA = [x|x — daviid, Aug 01 '13 at 11:14
Maybe it can be transformed to run in constant space. Maybe not. You could ask a separate question about it. — firefrorefiddle, Aug 01 '13 at 11:19

Daniel Gratzer · Accepted Answer · 2013-08-01T10:38:12.850

The problem is that your list has the type [ IO () ] or "A list of IO actions". Since the IO is on the "inside" of out type we can't execute this in the IO monad. What we want instead is IO (). So a list comprehension isn't going to hack it here.

We could use a function to turn [IO ()] -> IO [()] but this case lends itself to a much more concise combinator.

Instead we can use a simple predefined combinator called mapM_. In the Haskell prelude the M means it's monadic and the _ means that it returns m () in our case IO (). Using it is trivial in this case

[appendFile "test2.txt" (show x)|x<-mylistA]

becomes

mapM_ (\x -> appendFile "test2.txt" (show x)) myListA

mapM_ (appendFile "test2.txt" . show) myListA

This will unfold to something like

appendFile "test2.txt" (show firstItem) >>
appendFile "test2.txt" (show secondItem) >>
...

So we don't ever have the whole list in memory.

score 2 · Answer 2 · answered Aug 01 '13 at 10:34

You can use the function sequence from Control.Monad to take a (lazily generated) list of IO actions and execute them one at a time

>>> import Control.Monad

Now you can do

>>> let myList = [1, 2, 3]
>>> sequence [print x | x <- myList]
1
2
3
[(),(),()]

Note that you get a list of all the return values at the end. If you want to discard the return value, just use sequence_ instead of sequence.

>>> sequence_ [print x | x <- myList]
1
2
3

score 0 · Answer 3 · answered Aug 01 '13 at 13:46

I just wanted to expand on jozefg's answer by mentioning forM_, the flipped version of mapM_. Using forM_ you get something that looks like a foreach loop:

-- Read this as "for each `x` in `myListA`, do X"
forM_ myListA $ \x -> do
    appendFile "test2.txt" (show x)

Haskell: save one listitem at a time to file

3 Answers3