Production Support Analyst Sr - Observability

Job Summary:

We are seeking a senior Production Support Analyst professional to join our team on a contract basis to support a multi-year observability and monitoring modernization initiative. The ideal candidate has deep, hands-on experience designing, deploying, and operating the Grafana stack (Grafana, Loki, Tempo, Mimir/Prometheus, Alloy) in enterprise environments. They will assess our current monitoring tool landscape, identify overlap and gaps, and assist the technical onboarding of the Grafana stack into our ecosystem.

Key Responsibilities:

* Assist in assessing the current monitoring and observability toolset to identify functional overlap, coverage gaps, and consolidation opportunities, and help document findings and recommendations.
* Assist with the design and implementation of the Grafana stack (Grafana, Loki, Tempo, Mimir/Prometheus, Alloy) for metrics, logs, and traces, including architecture, sizing, high availability, and retention strategy.
* Assist with integrating Grafana with existing enterprise tooling and data sources (e.g., Dynatrace, Splunk, xMatters) and with identity/SSO providers.
* Assist in building and standardizing dashboards, alert rules, SLIs/SLOs, and on-call notification routing in partnership with application and infrastructure teams.
* Support the instrumentation of applications and infrastructure using OpenTelemetry, exporters, and agents; help define telemetry standards and onboarding patterns for new teams.
* Support the migration of existing dashboards, alerts, and monitoring use cases from legacy tools to Grafana, and assist with decommissioning of redundant tools.
* Assist with automating deployment, configuration, and maintenance using infrastructure-as-code and CI/CD (e.g., Ansible, Helm, Terraform, Git-based pipelines).
* Assist with deploying and operating observability components on Kubernetes/Red Hat OpenShift and Linux (RHEL) platforms.
* Help apply security best practices and compliance requirements, including RBAC, secrets management, data retention, and auditability.
* Contribute to runbooks, architecture documentation, and training materials, and assist with knowledge transfer to ensure operational ownership by staff by the end of the engagement.

Qualifications:

* Requires minimum 7 years of job-related work experience in excess of degree requirements, including at least 3 years of hands-on experience with the Grafana stack in production.
* Bachelor s Degree in Computer Science, Information Systems, or a related field (Required).
* Or a combination of education and experience that provides equivalent knowledge to a major in such fields is required.

Required Skills:

* Proven, hands-on experience implementing and administering Grafana, Loki, Tempo, and Prometheus or Mimir at enterprise scale.
* Strong experience with PromQL, LogQL, and Grafana alerting.
* Experience with OpenTelemetry instrumentation and collectors.
* Experience with Kubernetes and Red Hat OpenShift, including deploying observability components via Helm or Operators.
* Scripting and automation skills (e.g., Bash, Python) and experience with infrastructure-as-code tools (e.g., Ansible, Terraform).
* Experience with CI/CD and Git-based workflows.
* Working knowledge of at least one commercial APM/monitoring platform (e.g., Dynatrace, Splunk) and experience conducting tool rationalization or migration.
* Linux administration skills (RHEL preferred).
* Solid understanding of networking, storage, and security concepts as they apply to monitoring and telemetry pipelines.
* Experience integrating with SSO/identity providers (e.g., Okta, OIDC/SAML) (preferred).
* Strong analytical and problem-solving skills.
* Strong written and verbal communication skills, with the ability to document clearly and present recommendations to technical and management audiences.

Shift: []

Start: []

EEO:

Mindlance is an Equal Opportunity Employer and does not discriminate in employment on the basis of Minority/Gender/Disability/Religion/LGBTQI/Age/Veterans.