Software Engineer (Data- AWS/Snowflake)

Position Summary

Translates and develops requirements into workable software solutions. Maintains and develops programs for use in business and IT automation. Incorporates various accepted methodologies to design software and applications at a critical level to project or release. May perform duties in one or more of the following disciplines: Open Systems, Mainframe, Application, Configuration

What You'll Do:

The Data portfolio is looking for a talented and motivated Software Engineer specializing in data engineering, ETL, and AWS cloud technologies. This role will involve the design, implementation, and optimization of data pipelines, data lakes, and data warehouse solutions. You will be responsible for working across various AWS services, ensuring the seamless extraction, transformation, and loading of data, while maintaining best practices in data management and cloud solutions.

• Develop, deploy, and maintain ETL pipelines using AWS services such as Glue, Lambda, and Step Functions
• Implement data partitioning and manage Parquet file formats for optimized data storage and retrieval.
• Design, build, and manage data lakes and data warehouses using AWS S3, Redshift, and Snowflake.
• Perform data transformations using PySpark and AWS Glue.
• Integrate various AWS services like DynamoDB, RDS, SNS, and EventBridge for building data solutions.
• Ensure high availability and performance of data solutions using Redshift Spectrum and Athena.
• Work with Kafka for real-time data streaming and processing.
• Use API Gateway for managing APIs related to data integrations.
• Collaborate with cross-functional teams to define technical requirements and provide data-driven solutions.

What Makes You Stand Out:

• 3-5 years of experience in software engineering and data engineering.
• Proficiency with AWS services: S3, Redshift, Lambda, Glue, Step Functions, Athena, DynamoDB, RDS, EventBridge, SNS, and API Gateway.
• Strong experience in ETL concepts and tools, particularly AWS Glue and Informatica.
• Hands-on experience with PySpark and data partitioning using Parquet files.
• Familiarity with Kafka for data streaming.
• Experience with data warehouse and data lake concepts.
• Proficiency in SQL and working with databases such as Redshift, Snowflake, and RDS.
• Basic understanding of Docker for containerization and Terraform for infrastructure as code (IaC).
• Experience using version control tools like Git.
• Strong problem-solving skills and the ability to work in an agile environment.

Preferred Qualifications:

• Experience with Snowflake for data warehousing.
• Knowledge of Redshift Spectrum and Athena for querying large datasets.
• Understanding of DevOps practices and CI/CD pipelines.

Duties and Responsibilities

• Develops program logic for new, complex applications or analyzes and modifies logic in existing complex applications. Accurately delivers complicated code, configuration changes and enhancements.
• Contributes to releases with high quality code and unit tests, having challenges in complexity or deadlines. Implements and verifies a complete unit test.
• Contributes to design documents.
• Improves stability of platforms, applications, services and systems.
• Effectively communicates with customers, teammates or other stakeholders, and may function as lead or point of contact. Shares knowledge of systems and mentors or trains others.
• Develops and manages relationships with diverse groups of stakeholders at multiple levels.

The first five duties listed are the functions identified as essential to the job. Essential functions are those job duties that must be performed in order for the job to be accomplished.

This position description in no way states or implies that these are the only duties to be performed by the incumbent. Employees are required to follow any other job-related instruction and to perform any other duties as requested by their supervisor, or as become clear.

Competencies

Qualifications

• Bachelor's degree in IT, MIS, Business or related field, plus five years' related experience, or;
• Associate's degree in a technical field, plus seven years' related experience, or;
• High School Diploma plus nine years' related experience

Physical Requirements