Scalepex
AWS Data Engineer - Fully Remote - US Only
Plano, Texas
No sponsorshipDetected 4 days ago
PythonDistributed SystemsDynamoDBRedshiftAWSPandasAirflowData EngineeringAI OrchestrationForecastingCommunicationCollaborationProblem Solving
About the role
- We are seeking an experienced AWS Data Engineer with a strong background in building scalable data solutions and expertise in utilities-related datasets.
- The ideal candidate will have at least 5 years of experience in data engineering, a deep understanding of distributed systems, and proficiency with AWS services and tools like Step Functions, Lambda, Glue, and Redshift.
Responsibilities
- Design and Build Data Pipelines: Develop scalable, reliable data pipelines using AWS services (e.g., Glue, S3, Redshift) to process and transform large datasets from utility systems like smart meters or energy grids.
- Data Integration and Transformation: Implement ETL/ELT processes using PySpark, Python, and Pandas to clean, transform, and integrate data from multiple sources into unified datasets.
- Serverless Application Development: Use AWS Lambda functions to build serverless solutions for automating data processing tasks.
- Data Modeling for Analytics: Design data models tailored for utilities use cases (e.g., energy consumption forecasting) to enable advanced analytics
- Optimize Data Pipelines: Continuously monitor and improve the performance of data pipelines to reduce latency, enhance throughput, and ensure high availability.
- Ensure Data Security and Compliance: Implement robust security measures to protect sensitive utility data and ensure compliance with industry regulations.
- Develop scalable, reliable data pipelines using AWS services (e.g., Glue, S3, Redshift) to process and transform large datasets from utility systems like smart meters or energy grids.
- Implement ETL/ELT processes using PySpark, Python, and Pandas to clean, transform, and integrate data from multiple sources into unified datasets.
- Use AWS Lambda functions to build serverless solutions for automating data processing tasks.
- Design data models tailored for utilities use cases (e.g., energy consumption forecasting) to enable advanced analytics
Requirements
- Minimum of 5 years of experience in data engineering
- Proficiency in AWS services such as Step Functions, Lambda, Glue, S3, DynamoDB, and Redshift.
- Strong programming skills in Python with experience using PySpark and Pandas for large-scale data processing.
- Hands-on experience with distributed systems and scalable architectures.
- Knowledge of ETL/ELT processes for integrating diverse datasets into centralized systems.
- Familiarity with utilities-specific datasets (e.g., smart meters, energy grids) is highly desirable.
- Strong analytical skills with the ability to work on unstructured datasets.
- Knowledge of data governance practices to ensure accuracy, consistency, and security of data.
- Strong experience in AWS data engineering
- Ability to work independently
Nice to have
- Use AWS Step Functions to orchestrate workflows across data pipelines
- experience with Airflow is acceptable but Step Functions is preferred.
- Workflow Orchestration: Use AWS Step Functions to orchestrate workflows across data pipelines
Company info
- ❋ Why Scalepex?
- Scalepex is a dynamic services firm specializing in providing solutions for premium brands like Nike, Pepsi, Toyota, Virgin and Walgreens.
- Our mission is to connect prominent market leaders with top-tier professionals from around the world, fostering collaboration, efficiency, and growth.
- ❋ Take your portfolio to the next level by working with one of our fastest growing clients.
- Join the Innovation Frontier at Scalepex!
Apply directly at Scalepex →Create a free account for alerts like thisView Scalepex immigration profile
This listing is sourced directly from Scalepex's careers page and normalized into a canonical job model.