Data Engineering/Team roles and expectations

From Wikitech

Data Engineer

Collaboration

  • Participate in technical discussions and agile team rituals
  • Help to plan technical tasks and projects
  • Foster clear and open communication within the team
  • Raise issues proactively
  • Seek ways to improve how we work

Implementation

  • Write code to implement technical solutions to business requirements. Develop, test and maintain code, vet data, document tools and pipelines, troubleshoot, etc
  • Participate in evaluations - assessment, spikes
  • Solve issues and develop workarounds and alternative solutions
  • Build solutions for other teams, understanding their user experience and help develop solutions to meet their needs
  • Review and approve other team members' code to ensure high quality standards are met
  • Document work in phabricator, as well as DataHub and Wikitech when applicable

Support

  • Supporting end-users with the tooling related to data we provide them
  • Respond to questions that our users have including questions on the team Slack channel
  • Proactively seek and offer help to other team members
  • Maintenance/Operations
  • Troubleshooting operational incidents related to data products
  • Deploy code
  • Participate in Ops week
  • Seek ways to improve operational excellence

Learning

  • Responsible for own personal development plan with support from the manager
  • Share knowledge, write documentation, present, mentor
  • Gain expertise around tools that we are using and seek ways to improve how we work and use the tools
  • Drive innovation by researching novel solutions and trends, participating in hackathons, running POCs and sharing lessons learned

SRE

Collaboration

  • Participate in technical discussions and agile team rituals
  • Help to plan technical tasks and projects
  • Foster clear and open communication within the team
  • Raise issues proactively
  • Seek ways to improve how we work

Implementation

  • Write code to implement technical solutions to business requirements. Develop, test and maintain code, vet data, document tools and pipelines, troubleshoot, etc
  • Automate Infrastructure, configuration management, deployment, performance and observability
  • Participate in evaluations - assessment, spikes
  • Solve issues and develop workarounds and alternative solutions
  • Build solutions for other teams, understanding their user experience and help develop solutions to meet their needs
  • Review and approve other team members' code to ensure high quality standards are met
  • Develop and implement services that support the DE team, helping DE with critical issues
  • Document work in phabricator

Support

  • Supporting end-users with the tooling related to data we provide them
  • Respond to questions that our users have including questions on the team Slack channel
  • Proactively seek and offer help to other team members
  • Give operational support to non-sre team members
  • Supporting the core and other embedded SRE teams with cross-team requirements

Operational Excellence

  • Troubleshooting operational incidents related to data products
  • Deploy code
  • Participate in Ops week
  • Seek ways to improve operational excellence
  • Facilitate hardware provisioning and timely procurement
  • Monitoring and supporting Data Engineering Systems
  • Implementing of strategies that increase system reliability and security

Learning

  • Responsible for own personal development plan with support from the manager
  • Share knowledge, write documentation, present, mentor
  • Gain expertise around tools that we are using and seek ways to improve how we work and use the tools
  • Document knowledge gained after incidents
  • Drive innovation by researching novel solutions and trends, participating in hackathons, running POCs and sharing lessons learned

Tech Lead

  • Same as DataEngineer + SRE
  • Work closely with engineering, project and product managers to ensure that the solutions are well-matched to business requirements, and project delivered on time

Greater emphasis on:

  • Help plan technical tasks and projects, and support decision making
  • Help plan and prioritize projects and the general project direction
  • Help/mentor other developers and knowledge transfer
  • Conduct detailed assessments to make the right decisions with the team
  • Actively seek ways to improve the way the team works
  • Help delegate and assign work to team members
  • Review and approve other team members' code to ensure high quality standards are met
  • Sharing knowledge with the team through regular sessions
  • Providing feedback and mentorship to other team members, regarding specific areas of competence

Engineering Manager

Team Happiness

  • Responsible for well being and engagement of the team
  • Fostering clear and open communication within the team to maximise productivity
  • Help define team norms
  • Ensure clarity of roles and responsibilities
  • Cultivate a healthy team culture of trust, collaboration and learning
  • Recruiting and hiring
  • Individual team member development planning and career development
  • Setting performance goals
  • Conduct ITCs
  • Setting team long-term goal
  • Facilitate important team conversations

Team Productivity

  • Supports team productivity by ensuring the work is well planned, appropriately scoped and the team has the resources and tools to deliver on the commitments
  • Set priorities in collaboration with the project and product manager
  • Review engineers' code to ensure high quality standards
  • Reviewing and improving upon the processes we carry out as a team
  • Has a strong engineering background. Can write code, but is not required to be effective
  • Balances innovation, evolution and addressing technical debt
  • Supports technical decisions and code review
  • Balances the competing demands of cost, quality, and time, and making trade-off decisions as necessary

Process

  • Facilitates team meetings and other collaboration methods
  • Plan, design, and oversee projects
  • Helps formalize and improve engineering processes
  • Helps unblock engineers, facilitates team collaboration
  • Ensures uptime and ongoing maintenance of existing systems
  • Focus on continuous improvement

Collaboration

  • Works closely with external stakeholders to plan allocation of resources
  • Understands and advocates for customer success
  • Co-owns the roadmap with the PM
  • Responsible for the technical delivery of the product roadmap

Mentoring

  • Mentoring and coaching
  • Staying current with industry trends, technology developments, and new engineering methodologies, and incorporating this knowledge into the team's work
  • Facilitate knowledge sharing in the team
  • Identify and support learning opportunities

Director

Strategy & Vision

  • Develop a data strategy and vision in alignment with the broader WMF strategy and APP
  • Communicates strategy in the context of high level organizational priorities and strategy
  • Transmit the vision of the teams
  • Representing group at APP - help shape expectations vs reality in terms of things the foundation wants to do
  • Helps define the OKRs, responsible for delivery of OKRs
  • Stay current with industry trends, technology developments, and new product management methodologies, and incorporate this knowledge into the product strategy.
  • Assist senior management with the development and implementation of a data strategy
  • Approve initiatives and plans, roadmaps making sure they are aligned with overall plan and vision
  • Works across the department and organization to identify common themes, facilitate collaboration and deliver on common goals
  • Working closely with senior management to design and implement suitable policies and practices

Culture

  • Cultivate a healthy team culture of trust, collaboration and learning both at the group and department level
  • Responsible for well being and engagement of the group
  • Fostering clear and open communication within the teams to maximise productivity
  • Help define norms and processes
  • Ensure clarity of roles and responsibilities between teams and individuals
  • Encourages operational excellence
  • Ensures the right balance of innovation, evolution and addressing technical debt

Engineering Delivery

  • Budget owner for the team, ensures that the team(s) are appropriately resourced and work scoped accordingly
  • Ultimate arbiter on decisions required by the team, both technical and procedural
  • Assists team members to find help and solutions where none is apparent
  • Manages the shape and size of the team to ensure that it meets the needs of the business
  • Allocates resources
  • Takes responsibility for aspects of information security and users' privacy as they relate to our estate.
  • Protects the teams and empowers them to be experts in their work
  • Help removing team-level blockers
  • Listen to ICs and provide solutions to their potential issues
  • Identifying need of and driving hiring loops

Communication

  • Representing the Data Engineering team in front of the WMF and the community
  • Communicating & synchronizing with other Directors and senior leadership
  • Raising flags when things are not as they should be
  • Communicates high level organizational priorities and strategy
  • Communicates strategy in the context of high level organizational priorities and strategy

Project Manager

Project Delivery

  • Be an integral part in project planning, scheduling, facilitating, resourcing
  • Plan and break work into tasks, helping scope work appropriately
  • Determine key goalposts like project scope, timeline, and budget estimates
  • Ensure on time delivery
  • Facilitate resolution of blockers, surface issues
  • Track Progress. Report progress to stakeholders
  • Determine and coordinate team dependencies
  • Manage work through the project management software

Agile and other Processes

  • Run team rituals including scrum, planning, retrospective, demos
  • Define and Improve processes
  • Educate team to follow the process
  • Derive continuous improvement / Identify Improvements
  • Follow through on retro feedback and todo

Communication

  • Coordinate work with other teams and stakeholders
  • Communicate with leadership
  • Communicate about process changes
  • Foster clear and open communication within the team to minimize project risk
  • Make sure teams are collaborating well, and staying motivated and on schedule

Product Manager

Product Strategy & Vision

  • Develop product vision, strategy, and roadmap, and communicate it to stakeholders
  • Analyze customer needs, market trends, product landscapes etc. to inform product strategy
  • Requirement gathering and identifying gaps
  • Aligning stakeholders around the vision for the product
  • Communicates product vision in the context of high level organizational priorities and strategy
  • Stay current with industry trends, technology developments, and new product management methodologies, and incorporate this knowledge into the product strategy.
  • Assist senior management with the development and implementation of a data strategy
  • Understand the needs of the free knowledge movement, interpreting those needs as they apply to the possible outputs of the team they're product managing, and collaboratively breaking down those needs into tasks.
  • Work with cross-functional teams to develop and pursue product strategy
  • Helps define the OKRs

Planning and Execution

  • Collaborate with engineering director, tech leads and engineering team to define the 'how' of the product vision
  • Developing a consensus on what to work on with the team, stakeholders, directors, engineers, etc
  • Works with PjM, EM and engineering team to scope the work in an agile manner
  • Determine key goalposts like project scope, timeline, and budget estimates
  • Work closely with engineers to surface any potential problems in project delivery
  • Establishing and communicate priorities
  • Make tradeoffs in product decisions to accommodate timely delivery
  • Develop launch strategies to effectively launch new products/features related to the foundation.
  • Define and communicate key metrics for product success
  • Review and improve processes, to drive quality and efficiency.
  • Understanding and representing user needs
  • Reviewing and signing off on completed work

Communication

  • Convey product requirements and customer needs to the team
  • Collaborate and communicate extensively with leadership and stakeholders
  • Responsible for internal and external stakeholder communication about new features and services
  • Foster clear and open communication within the team to maximize productivity
  • Negotiate with EM/Director to ensure a right balance of customer facing and internal technical work
Retrieved from "https://wikitech.wikimedia.org/w/index.php?title=Data_Engineering/Team_roles_and_expectations&oldid=2061349"