Data Engineer with IRS MBI Clearance

Job Title: Data Engineer Location: Remote. Duration: Full-Time. NEED IRS MBI Clearance with IRS GFE Position Overview: We are seeking an experienced Data Engineer to design, build, and maintain scalable data processing pipelines using modern big data technologies. The ideal candidate will have strong development skills and experience with distributed computing frameworks. Required Skills & Qualifications: Bachelor's degree in Computer Science, Engineering, or related technical field 5+ years of software development experience Strong proficiency in Python, Java or Scala programming languages Extensive hands-on experience with Apache Spark, including: Spark SQL Spark Streaming and its data sources KAFKA, CDC feed, etc Performance optimization and tuning Data transformation and processing Hands on Experience working with cloud-based Spark platforms (Databricks, AWS EMR, AWS Glue) Strong understanding of Hive, Unity catalog, Glue catalog Hands on Experience working with any data quality framework/tools Hands on Experience working with Observability and monitoring for data processing pipelines Strong understanding of distributed computing concepts Proficiency in version control systems (Github/GitLab/Bitbucket) Technical Requirements: Experience building and maintaining high-volume data processing pipelines Knowledge of data modeling and ETL / ELT best practices Familiarity with SQL and NoSQL databases Understanding of data warehouse concepts and dimensional modeling Preferred