# Infinity

- **Event:** [Meta Global AI Developer Hackathon](https://cerebralvalley.ai/e/meta-ai-developer-hackathon)
- **When:** Oct 26, 12:00 PM – Nov 4, 9:00 PM (EST)
- **Where:** Online
- **Team:** [Syed Muhammad Taha Hassan Shah](https://cerebralvalley.ai/u/tahahassan)
- **GitHub:** https://github.com/tahahssn/sanitizepy
- **Demo video:** https://drive.google.com/file/d/12ChXTayoXZU3qU6CyAzW9xgITfU1T_ut/view?usp=sharing
- **Gallery:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery
- **Page:** https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2593

Sanitizepy. Explainable Data Cleaning and Data Quality for Python

sanitizepy is an open-source Python library for making data cleaning more explainable and understandable. It allows developers, data analysts and other data practitioners to examine, clean, validate and transform messy datasets without using various scattered scripts in pandas.

The library provides a framework to examine data quality before applying any transformations to datasets. The library analyses data, calculates dataset health scores, detects and explains data quality problems along with their severity level and evidence. The library doesn't only provide transformations to the dataset but also allows users to get insights into problems and understand how these problems may be solved by applying some cleaning operations.

Users may use the library to generate cleaning plans, inspect individual transformations, select the operations to apply and preview what will happen as a result of those changes before transforming a dataset. The ability to review potential changes to the dataset before applying them allows the users to have more control over the cleaning process.

sanitizepy provides a set of tools that allow detecting such problems as missing values, incorrect data types, inconsistent text normalization, bad encodings, near duplicates, dataset profiling, data contract violations, anomalies detection and text quality analysis. These tools cover various aspects of data preparation from detecting structural problems in datasets to analyzing problems in textual data.

Being focused on transparency and traceability, every time you run any cleaning operations, the library creates an audit log which contains information about the operations you have applied and the affected rows and columns. Moreover, before and after metrics allow you to know how your dataset has changed as a result of the cleaning operation. Being combined with deterministic transformations, the process of cleaning becomes easier to review, repeat and maintain.

The library provides an easy-to-use API based on pandas which allows to easily solve various data quality and cleaning problems, and an advanced API which allows more control over individual operations and full cleaning workflow.

The main aim of the library is not just to provide a way to apply transformations to the dataset but also to allow users to understand their data, make valid cleaning decisions, validate the results and maintain the records of all performed changes.

## More from Meta Global AI Developer Hackathon

- [Eucher Abatti](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2590)
- [Resonance Labs](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2591)
- [StormGate-Zero](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2592)
- [Pawprint Club](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2594)
- [Jeffery Nketiah](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2595)
- [Fight Club](https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2596)

---

Markdown version of https://cerebralvalley.ai/e/meta-ai-developer-hackathon/hackathon/gallery/2593. Site index for agents: https://cerebralvalley.ai/llms.txt · full text: https://cerebralvalley.ai/llms-full.txt