Metadata-Version: 2.4
Name: triton-deep
Version: 0.2.3
Summary: A better framework for your Spark data transformation
Home-page: https://cicd-gitlab.telkomsel.co.id/deep/triton
Author: Data Engineering Expert Pool
License: MIT License
Project-URL: Bug Tracker, https://cicd-gitlab.telkomsel.co.id/deep/triton/issues
Classifier: Development Status :: 3 - Alpha
Classifier: License :: OSI Approved :: MIT License
Classifier: Programming Language :: Python :: 3.6
Classifier: Programming Language :: Python :: 3.7
Classifier: Programming Language :: Python :: 3.8
Classifier: Topic :: Software Development :: Code Generators
Requires-Python: >=3.6
Description-Content-Type: text/markdown
License-File: LICENSE
Requires-Dist: Jinja2>=3.0.2
Requires-Dist: python-dateutil>=2.8.2
Requires-Dist: python-json-logger>=2.0.2
Requires-Dist: pytz>=2021.1
Requires-Dist: PyYAML>=5.4.1
Requires-Dist: ruamel.yaml>=0.15.35
Requires-Dist: ruamel.yaml.clib>=0.2.6
Requires-Dist: shapely>=1.7.1
Requires-Dist: croniter>=1.0.10
Requires-Dist: gpudb>=7.0.19.0
Requires-Dist: cookiecutter>=1.7.3
Requires-Dist: numpy>=1.18.0
Requires-Dist: pandas>=1.1.5
Requires-Dist: paramiko>=2.9.2
Requires-Dist: psycopg2-binary>=2.9.2
Requires-Dist: PyMySQL>=1.0.2
Requires-Dist: pydantic>=1.9.0
Requires-Dist: pytest>=6.2.0
Requires-Dist: sqlalchemy>=1.4.29
Requires-Dist: sql-metadata>=2.2.2
Requires-Dist: sqlparse>=0.4.2
Requires-Dist: gpudb>=7.0.19.0
Dynamic: author
Dynamic: classifier
Dynamic: description
Dynamic: description-content-type
Dynamic: home-page
Dynamic: license
Dynamic: license-file
Dynamic: project-url
Dynamic: requires-dist
Dynamic: requires-python
Dynamic: summary

## What is Triton?

Triton is a Spark framework for data transformation, focusing on simplicity, portability, and concern separation. With Triton, we can:

- Separating logic, resource, and catalog concerns
- Scheduler-agnostic, should not be tightly coupled with any scheduler tools​
- Easy to use, user can focus on logic rather than technical details
- Hiding code complexity
- Logic & scheduler details in the same place


## Why do we need Triton?
- Maintaining tens or hundreds of PySpark Scripts are hard
- No Standard for PySpark scripts developments
- No separation of concern for logic, spark configurations, and feed details
- Completely separated place/repo for logic & scheduler

## Triton Architecture

//TODO


## How do I install Triton?

To install Triton from the telkomsel's jfrog articatory (pip), simply run:

```
pip install triton-deep
```

It is advisable to use `virtualenv` before you run the `pip install` command.

If you are using CDSW, you are good to go

If you are using Edge Node, you should create a new python `virtualenv` as follow:
- `/opt/cloudera/parcels/Anaconda-5.1.0/bin/python -m venv venv`
- `source venv/bin/activate`

## How do I create a new Triton Project?

To create a new Triton project, we can run:

```
triton create -p <PROJECT_NAME>
```

This command will create a new directory with some boilerplate code & config example in it
