I need to scan very large JSONL files efficiently and am considering a parallel grep-style approach over line-delimited text.
Would love to hear how you would design it.
I need to scan very large JSONL files efficiently and am considering a parallel grep-style approach over line-delimited text.
Would love to hear how you would design it.
Could u use an already parallelized solution like ripgrep? I think someone else also mentioned putting it in a database, that shouldnt be too bad either.