Data Engineer (Quantexa)

Implement data transformation, aggregation, and enrichment processes to support various data analytics and machine learning initiatives Collaborate with cross-functional teams to understand data requirements and translate them into effective data engineering solutions Design, develop, and implement Spark Scala applications and data processing pipelines to process large volumes of structured and unstructured data Integrate Elasticsearch with Spark to enable efficient indexing, querying, and retrieval of data Optimize and tune Spark jobs for performance and scalability, ensuring efficient data processing and indexing in Elasticsearch Implement data transformations, aggregations, and computations using Spark RDDs, DataFrames, and Datasets, and integrate them with Elasticsearch Develop and mai…