Migrated legacy SSIS workflows to Google Cloud by building a cloud-native ETL pipeline with Apache Beam and Cloud Dataflow, enabling batch ingestion of over 160K records with scalable processing and ensuring data accuracy post-migration.
1roles
38skills
1education
3credentials
About
Data Engineer with over a year of hands-on experience in building ETL pipelines and cloud-based data platforms. Proficient in leveraging technologies like PySpark, Databricks, and AWS to deliver high-quality data solutions.
Experience
Data Engineer
2GBR Software Private Limited
May 2025 – Present · Pune, Maharashtra
Nidhi built and maintained over 10 batch ETL/ELT pipelines using PySpark and Databricks, enhancing throughput by 30%. She optimized Spark workloads, reducing job runtime by 25%, and designed AWS S3-based data lake workflows processing 30-50 GB of data daily. Her implementation of schema validation and automated data quality checks achieved over 99% data accuracy on 5 million records processed daily.
Education
S.B. Jain Institute of Technology, Management & Research
B.Tech, Artificial Intelligence and Data Science
2021 – 2025
Skills
PythonSQLPySparkApache SparkSpark StreamingApache KafkaDatabricksDelta LakeAWSGCPBigQueryCloud DataflowGCSApache BeamMySQLETLELTData ModelingStar SchemaBatch ProcessingReal-Time StreamingData WarehousingData LakeData QualityApache AirflowGitGitHubJupyter NotebookVS CodeAWS S3AWS EMRGCP BigQueryGCP Cloud DataflowETL/ELT PipelinesGit/GitHubHive MetastoreHDFSSpark SQL