It seemed impossible in the moment but it was true: Only one Data Engineer was presenting at the summer 2022 Digitalizing Mining in North America conference in Toronto: Meera Nair Meena. Many Data Scientists, as well as other data-focused mining professionals, were present.
For an event devoted to harnessing data in the mining sector, this was striking. Data science is, of course, crucial but its practitioners can get lost, build models on incomplete information, or spend too much time working outside their purview to do their own jobs as well as possible. Yet, Data Scientist remains the most well-known and well-represented role in companies looking to make the most of their data.
We spoke with Meera about this trend, why data engineering is so important for data-driven mining companies, and how to pursue a career in this still under-respected area.
At the start of her DMNA presentation, Meera mentioned that she seemed to be the only Data Engineer present. Afterwards, many people asked her about the difference between data science and data engineering, which is encouraging; Meera said she “really appreciates those questions.” Others agreed with her, saying their organizations either don’t have enough—or even any—Data Engineers.
However, some participants still wondered, “Who needs data governance?” Only anyone who’s serious about using company data! Which should be everyone.
Meera asserted that this field may be unknown or less appreciated than it should be; but it’s essential for any organization—mining or otherwise—that relies on data to make smart operational and business decisions.
Data Engineers collect, store, and clean data so Data Scientists and other staff know they’re working with the most current, accurate resources. As Meera noted, “Being a Data Engineer involves a compelling blend of hardcore engineering work, clear and strategic communication with other experts along company data paths, and opportunities to help guide the entire organization’s activities;” DEs do this using “their unique, comprehensive understanding of all the information floating around.”
Herself highly involved in all such tasks, Meera affirmed the value of the “pure” engineering work required; she also underlined the collaboration and interactions that naturally result from this work as essential to most effectively using company data and human resources.
When performing data engineering work, Meera suggested that DEs “Always ask, ‘Why are we doing this? Why does the end user need this?’ Ask why a lot; don’t just do something because it’s been asked of you, without knowing the value. Ask at least two or three why’s, especially why the person or team requesting it needs it; have that kind of empathy for end user value.”
One set of users is, of course, the Data Scientists who take over working with data once, ideally, Data Engineers have sorted, cleaned, and verified it. Meera insisted, and we agree, that “Both need to be present. Data engineering for the data side; Data Scientists for the process side.”
Indeed, not only should both types of experts be established and supported in your data-driven organization, but they and everyone around them should also clearly understand what each is responsible for, why, and when.
She also advised:
And, perhaps our favorite, because it speaks to both efficiency and smart resource allocation:
With these kinds of values, understanding, and infrastructure in place for company data management and use, data engineering can improve every aspect of even the most complex organizations.
As a woman working on the corporate side of mining, she said, “There are notions everywhere about women that need to be overcome; for example, people saying I’m a good people person because it’s in my genes.” (Insert very professional eye roll here.)
Despite minor annoyances like this, Meera enjoys deep job satisfaction but, she said, “Even more than the job itself, I love the domain.” From her perspective, mining needs Data Engineers much more than tech companies do.
And she’s clear in her purpose, even if the industry and her title may not be “fancy.” She accepts that, as a DE, “You need to be an evangelist for this work,” since its importance is not yet fully appreciated.
Meera is comfortable leading this charge: “I love the challenge” and “the last four years have been the happiest in my career.” Data engineering itself is, of course, sufficiently stimulating on its own but she also likes “the non-tech challenges; addressing them makes it clear why it’s important. This is where I feel I’ve made the most impact.”
As someone who didn’t begin her career as a DE, Meera noted, “There are many ways to come to data engineering work; it doesn’t need to be a straight line from first-year university to your first DE role. Data science is always in the spotlight so many go there for their careers.”
She suggested that young professionals “Try both and see where you fit. Understanding data science is key so you can speak the same language.” We agree; diversity in education and experience will only benefit Data Engineers, the other data experts they work with, and the organizations employing them.
Meera studied Electronics and Communications Engineering at university and began working in software engineering after her degree. While at university, she was very strong in electronics but not in computer science. At her first job, she received comprehensive computer science training—so comprehensive, in fact, that she learned enough to become a trainer.
“This was the biggest turning point in my career,” Meera said, because “there were opportunities to move to whatever team interested me.” She chose to go to the Big Data team, an area with a lot of innovative and practical applications within any number of industries.
Meera has been in the data space 12 years; she now works for Rio Tinto, where she took on her first official data engineering position. In her experience, “Being a Data Engineer means you have to tame your ego sometimes; Data Scientists get more of the spotlight because they build the models.” It’s just part of the work, she said, that “The DE has to design the solution and then step back.”
She used to find this quieter position in the grand work of wrangling data somewhat disappointing; however, she’s since “gone past these feelings,” confident about the essential contributions only Data Engineers can make.
And, she noted, things are improving: When she joined Rio Tinto, she was the only DE but now the company has a full DE team. When asked what advice she’d give to young DEs just starting out, Meera suggested: “Be a good software engineer. Develop data acumen and common sense. Focus on data, not on tech.”
From her perspective, while “other kinds of tech companies pay more,” in mining, “work-life balance is better because people and companies care about people and safety; other companies focus on revenue.” In fact, there are many great benefits in working as a DE in mining but, she admitted, “It’s hard to advertise quality of life.”
Meera gives this an honest shot, however, when she hires interns; this is crucial, we think, in a not quite post-pandemic world where “quiet quitting” has followed the “great resignation” as people decide that life has too many uncertainties to waste time in jobs that aren’t amazing.
With her interns, she’s careful to “give them a good experience so they tell their peers. I give them opportunities to work on things they like; I treat them like full-time employees and they work on real projects.” She never has them taking notes or performing other simple administrative work, which still (unwisely) occurs in many kinds of jobs and across sectors.
Whatever you studied or did not study at university, whatever you’re doing for a living now, data engineering is something to explore. For work that is increasingly essential, in demand, pays well, and requires using both analytical and communication skills, it really can’t be outdone.
And while it may be a somewhat lonely role to begin, we think the importance of data engineering is well on its way to garnering the widespread respect—and associated demand—it deserves.
Speak to Our Experts
Connect with a 3AG Systems expert today and start your journey towards efficient and effective data management.
