Description:

Quorum is pioneering the development of Generative AI technology that revolutionizes data analysis and utilization in government affairs.
The latest product, Quorum Copilot, navigates vast political data, including 120,000 bills from 50 state legislatures, over 100,000 social media posts, 10,000 documents, and more than 100 hours of testimony from Capitol Hill.
Quorum Copilot aims to enhance productivity, transform workflows, and save time for professionals in public affairs and policy advocacy.
As a Senior Data Engineer, you will lead the effort to build and scale the data infrastructure for the AI analysis tool.
You will work alongside software engineers, product managers, and product designers to ensure data systems are scalable and efficient.
Key responsibilities include architecting and implementing scalable data pipelines, designing robust data processing systems, leading cloud-based deployments in AWS, and innovating data architecture to support dynamic needs.

Requirements:

A Bachelor's or Master's degree in Computer Science, Engineering, or a related field is required.
A minimum of 5 years of experience in data engineering with a proven track record of developing and scaling data-driven products is necessary.
Expertise in building data pipelines and architectures, with experience in AWS cloud services (EC2, EMR, RDS, Redshift) is essential.
Proficiency in big data tools (Hadoop, Spark, Kafka) and machine learning frameworks (TensorFlow, PyTorch) is required.
At least 3 years of experience with Python is needed.
Deep knowledge of SQL and NoSQL databases, as well as workflow management tools (Azkaban, Luigi, Airflow), is necessary.
Experience or working knowledge of vector databases and advanced Retrieval-Augmented Generation (RAG) systems is preferred.
Candidates should have a strong interest in learning new development languages, tools, and platforms.
A highly motivated self-starter attitude and a desire to contribute to a team from day one are essential.
Candidates should be engaged learners who want to expand their skills and tackle challenging problems.