An example of how one can partition a large file and process it in parallel. The core function is `partition`. Also look at how multiprocessing's starmap_async is used to get the result.
Stefano Mazzucco
fb65c2920c
Remove chunksize from starmap_async
|
il y a 6 ans | |
|---|---|---|
| .gitignore | il y a 6 ans | |
| LICENSE | il y a 6 ans | |
| README | il y a 6 ans | |
| partition.py | il y a 6 ans |
README
An example of how one can partition a large file and process it in parallel.
The core function is `partition`. Also look at how multiprocessing's
starmap_async is used to get the result.
The core function is `partition`. Also look at how multiprocessing's
starmap_async is used to get the result.