Ayursed Erdenebulag

Data stack: Azure, Snowflake, DBT, Databricks, Github

Data Engineer role:

• Being the only data engineer in the team, structured data platform from scratch using head quarter's guidance. Served data to Data analyst and Data Scientist.
• Efficiently inherited existing batch ETL process
• Proactively identified opportunities for process improvements, addressing inefficiencies on ETL processes. By optimizing python pandas operations, able to make 1 hour databricks process time 10 minute process time.
• Like engineering best practice, sat an efficient monitoring which worked for early detection of incidents. Analyzed root cause and actions to be made in Scrum manner.
• Designed scalable and maintainable data models such as Kimball data structure to support business intelligence initiatives and reporting needs.

Outside of Data Engineer role:

• For 6 month, led pipeline cloud migration project communicating with Stake holder, Data analyst, other region's Data engineer. Did task management and was able to successfully migrate the project.
• With Retool UI developer, Implemented a data mapping tool and delivered it to data stewards which would replace excel data maintenance and improved data quality.
• Project Outlet master: Designed the system architecture and implemented base structure of in-house data deduplication tool which is based on Retool UI app. Effectively collaborated with our team's Data scientist who'd help on identifying if the data across sources are same entity or not by leveraging string comparison algorithms and OpenAI language learning models. Also after releasing the tool, guided the data stewards by writing onboarding documents and communicating with them face-to-face.

Maintaining marketing support tool called 「Yahoo! Audience Discovery」. Which uses yahoo's big data. Doing data engineering and web maintaining role.

Data engineer role:

• Proposed new formula in advertisement's reach simulation which is easy to interpret, which decreased number of inquiry 20%.
• Reviewed Data Lake team's new data stream. In order to best fit to Yahoo! Audience Discovery product, request changes to data supply team if needed.
• Turned 6 people/month sized ETL fixing project into achievable increment tasks. Developed with 3 member and released in 2 months.
• Summary impact analysis on updated tables, communicate to sales team if renewing data have negative impact on revenue.
• Tuned HQL and HQL's configuration to increase speed. Removed reducer bottleneck 60% increase in one of main queries.

Web maintainer role:

• Develop and implement migration of current API server and batch server to CaaS(Containers as a Service), PaaS(Platform as a Service).
• Maintaining approximately 20 different backend batch component, 10 different backend API, front-end as scrum team.
• Add necessary improvement of current component to backlog and determine it's priority
• Design, develop, code review, test, release feature in Batch, API servers.

• Applied ML algorithm to HVAC system
• Worked with software development members to design and develop robust solutions to meet client requirements for functionality, scalability and performance.
• Tested and validated models for accuracy of predictions.
• Deployed into production environment.
• Upgraded, fine tuned model using increasing data set. Resulted 30% user claim reduction from initial model.
• Worked with stakeholders to develop quarterly roadmaps based on impact, effort and test coordinations

Developed system which automates ad-hoc aggregation.

Duties:

• Developing libraries on Python for batch server to interact with apache Hadoop and AWS s3.
• Testing, tuning HQL(Hadoop query language) to meet with business SLA.
• Built, tested and deployed highly available service.
• Interacting with Project manager and reporting other members progress.
• Pointed out technical risks of system to PM using probability, importance factor.

• Made prototypes of image recognition model using deep learning under given dataset
• Improved accuracy of newly prototype by alternating both neural network and data annotation