Polars for initial data analysis, Polars for production
Initial data analysis (IDA) has different goals than your final, production data analysis:
- With IDA you need to examine the initial data and intermediate results, check your assumptions, and try different approaches.
Exploratory data analysis has similar requirements. - Once you’re happy with your approach, and you’re ready to run the analysis in an automated manner, you care a lot more about speed and resource usage.
These different goals often benefit from different implementation strategies and tools—unless you have a sufficiently flexible tool like Polars, the super-fast dataframe library.
In particular, Polars has two fundamental APIs, each of which is useful in different situations:
- “Eager” mode, which is similar to how Pandas works, is well-suited for initial and exploratory data analysis.
- For production use, “lazy” mode often execute much