# -*- coding: utf-8 -*-
from setuptools import setup

packages = \
['structured_data_profiling',
 'structured_data_profiling.data_slicing',
 'structured_data_profiling.data_tests',
 'structured_data_profiling.expectations',
 'structured_data_profiling.preprocessor',
 'structured_data_profiling.profiler',
 'structured_data_profiling.utils']

package_data = \
{'': ['*']}

install_requires = \
['dateparser>=1,<2',
 'distfit>=1.4,<2.0',
 'great-expectations>=0.15.21,<0.16.0',
 'numpy>=1.21,<2.0',
 'pandas>=1.3.0,<2.0.0',
 'scikit-learn>=1.0,<2.0',
 'scipy>=1.7.2,<1.8.0']

setup_kwargs = {
    'name': 'structured-profiling',
    'version': '0.3.5',
    'description': 'A Python library to check for data quality and automatically generate data tests. ',
    'long_description': '\n## StructuredDataProfiling\n\nThe StructuredDataProfiling is a Python library developed to automatically profile structured datasets and to facilitate the creation of **data tests**.\n\nThe library creates data tests in the form of **Expectations** using the [great_expectations](https://www.greatexpectations.io) framework. Expectations are \'declarative statements that a computer can evaluate and semantically meaningful to humans\'.\n\nAn expectation could be, for example, \'the sum of columns a and b should be equal to one\' or \'the values in column c should be non-negative\'.\n\nStructuredDataProfiling runs a series of tests aimed at identifying statistics, rules, and constraints characterising a given dataset. The information generated by the profiler is collected by performing the following operations:\n\n- Characterise uni- and bi-variate distributions.\n- Identify data quality issues.\n- Evaluate relationships between attributes (ex. column C is the difference between columns A and B)\n- Understand ontologies characterizing categorical data (column A contains names, while B contains geographical places).\n\nFor an overview of the library outputs please check the [examples](./examples) section.\n\n# Installation\nYou can install StructuredDataProfiling by using pip:\n`pip install structured-profiling\n`\n# Quickstart\nYou can import the profiler using\n\n```python\nfrom structured_data_profiling.profiler import DatasetProfiler\n```\nYou can import the profiler using\n```python\nprofiler = DatasetProfiler(\'./csv_path.csv\')\n```\nThe presence of a primary key (for example to define relations between tables or sequences) can be specified by using the argument **primary key** containing a single or multiple column names.\n\nTo start the profiling scripts, you can run the profile() method\n```python\nprofiler.profile()\n```\nThe method generate_expectations() outputs the results of the profiling process converted into data expectations. Please note, the method requires the existence of a local great_expectations project.\nIf you haven\'t done so please run ```great_expectations init``` in your working directory.\n```python\nprofiler.generate_expectations()\n```\nThe expectations are generated in a JSON format using the great_expectation schema. The method will also create data docs using the rendered provided by the great_expectations library.\n\nThese docs can be found in the local folder ```great_expectations/uncommitted/data_docs```.\n\n# Profiling outputs\nThe profiler generates 3 json files describing the ingested dataset. These json files contain information about:\n- column_profiles: it contains the statistical characterisation of the dataset columns.\n- dataset_profile: it highlights issues and limitations affecting the dataset.\n- tests: it contains the data tests found by the profiler.\n\nThe process of generating expectations makes use of the great_expectations library to produce an HTML file contaning data docs. An example of data doc for a given column can be seen in the image below.\n\n<img alt="data docs example 1" src="https://raw.githubusercontent.com/Clearbox-AI/StructuredDataProfiling/main/examples/num_columns.PNG"/>\n\n\n# Examples\nYou can find a couple of notebook examples in the [examples](./examples) folder.\n# To-dos\nDisclaimer: this library is still at a very early stage. Among other things, we still need to:\n\n- [ ] Support more data formats (Feather, Parquet)\n- [ ] Add more Expectations\n- [ ] Integrate PII identification using Presidio\n- [ ] Optimise and compile part of the profiling routines using Cython\n- [ ] Write library tests\n',
    'author': 'Clearbox AI',
    'author_email': 'info@clearbox.ai',
    'maintainer': None,
    'maintainer_email': None,
    'url': 'https://github.com/Clearbox-AI/StructuredDataProfiling',
    'packages': packages,
    'package_data': package_data,
    'install_requires': install_requires,
    'python_requires': '>=3.7.1,<3.11',
}


setup(**setup_kwargs)
