- Create a high throughput Integration channel to support the extraction, Transformation and Loading of the data
- Use Apache Spark and other high performance Big Data Technologies to implement the necessary Data pipelines.
- Ingest data from a variety of sources using a fully automated consumption system.
- Build a common Data Quality framework to prune all the incoming data and make sure it complies with the required application quality requirements.
- Design and Develop a Key Value store based Enterprise Data Model to support analytics.
- Use the key value store to generate high performance consumption views.
- Create end-end data flows with proper parallelism and fault tolerance to run the data pipelines.
Bachelor’s degree in Computer Science or a closely related field