Job Description Our client is seeking a Databricks SME to support a federal end customer with a build and modernization effort. As a Databricks SME, the responsibilities include, but are not limited to: • Modernize the data warehouse environment by migrating the platform to Databricks. • Work with database developers and administrators across multiple product teams. • Serve as a data and technology expert across a broad and diverse set of mission critical applications. • Design, develop, and maintain robust and scalable data warehouse architectures and ETL/ELT data pipelines using Databricks, PySpark, Python, SQL. • Automate ETL/ELT data pipelines using Continuous integration, Continuous Deployment (CI/CD) tools and technologies. • Evaluating existing data sets and reporting architectures to identify strategic gaps and apply modern technologies to creatively achieve superior mission outcomes. • Analyze project-related problems and create innovative solutions involving technology, analytic methodologies, and advanced solution components. • Optimize and troubleshoot data pipelines and warehouse performance to ensure efficient and reliable data processing. • Actively participate in Agile Scrum sprint planning, artifact creation, sprint testing, regression testing, demonstrations, retrospectives and solution releases. We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to HR@insightglobal.com.To learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/. Skills and Requirements • Active Top Secret Clearance • 7+ years of professional experience working on complex data challenges in the areas of data architecture and engineering • 3-5 years of Databricks experience. Alternative/equivalent technologies: Significant experience with Snowflake, Google BigQuery, or Microsoft Azure Synapse Analytics will also be considered. • Proven expertise with Databricks, including extensive hands-on experience with PySpark, Python, SQL, Kafka, and Databricks notebooks. • Strong experience with data modeling techniques (e.g., dimensional modeling, data vault) and database design. • Experience building and optimizing data pipelines for batch and/or streaming data. • Experience with cloud platforms (e.g., AWS, Azure, GCP) and services related to data storage and processing (e.g., S3, ADLS). • Experience automating ELT data pipelines using Continuous integration, Continuous Deployment (CI/CD) tools and technologies. • Strong software development background using Agile or DevOps methods and deep familiarity with cloud-native technologies. • 5-10 years of DHS, DoD, or IC experience working in complex data environments, including the architecture and optimization of data schemas, terabyte-scale ETL, etc. • 5-10 years of experience applying a range of analytical techniques including statistical, geospatial, link, temporal, and predictive analysis, for DHS, DoD, or IC agencies. • 3-5 years of experience building and implementing artificial intelligence, neural networks, deep learning, or machine learning capabilities in software applications in a national security or academic environment. • Exposure to implementing or migrating to Cloud environments like Amazon Web Services (AWS) or Microsoft Azure. • Previous experience as an Enterprise-level Data Architect, Data Engineer, Data Scientist, or Data Analyst. • Ability to apply advanced principles, theories, and concepts, and contribute to the development of innovative principles and ideas.