SRE / DevOps / Platform Engineering
Reliable infrastructure, kept understandable.
I am Trinidad Marroquin, an SRE / DevOps / Platform Engineer focused on Kubernetes, infrastructure automation, observability, secrets management, and production operations for systems that need to be operated with confidence. I also bring technical leadership experience from leading a DevOps engineering team for three years.
I work on infrastructure that needs to be understandable, repeatable, and boring in production. My focus areas include Kubernetes, cloud platforms, vCenter, Terraform, Rancher, Packer, Vault, CI/CD, Linux systems, and observability.
Best Fit#
- SRE, DevOps, Platform Engineering, and Infrastructure Automation roles.
- Kubernetes and Rancher platform operations, especially environments that need clear upgrade, recovery, and evidence practices.
- Terraform, Packer, Vault, CI/CD, Linux, vSphere, and observability-heavy infrastructure.
- Teams that value practical runbooks, safer changes, useful alerts, and systems that can be debugged under pressure.
Start Here#
Focus Areas#
Cluster operations, Rancher management, upgrades, access patterns, and production readiness.
Terraform modules, reviewable plans, environment promotion, and safer operational changes.
Cloud platform operations, vCenter administration, VM lifecycle, templates, identity, and network foundations.
Packer image pipelines and CI/CD workflows that keep deployments repeatable and debuggable.
HashiCorp Vault operations, policy design, secrets engines, authentication methods, and operational guardrails.
Metrics, logs, alerts, dashboards, and incident follow-up focused on useful operating signals.
What This Site Is For#
- Practical notes on SRE and DevOps work.
- Project writeups with constraints, tradeoffs, and operational outcomes.
- Field notes for practical command references, troubleshooting checks, and operational judgment.
- Technical blog posts about Kubernetes, cloud platforms, vCenter, Terraform, Rancher, Packer, Vault, CI/CD, and observability.
- A current resume and professional summary.
During a vSphere CNS backlog, stopping the source of new CSI work can be safer than continuing to submit attach, detach, resize, and update requests into a stuck backend queue. The trap is that GitOps may immediately undo the pause.
Scaling a vSphere CSI controller Deployment to 0 is not a durable pause if Argo CD, an ApplicationSet, or a higher root Application owns it. The visible command succeeds, then self-heal restores the replicas and new controller pods start submitting tasks again.
...
A pod stuck in Pending because a PVC does not exist is not always a provisioning problem. Sometimes the data still exists in a retained PV and only the claim object is missing.
The safe recovery is to prove the PV is the intended backing store, then recreate the PVC with an explicit volumeName so it binds to that retained PV instead of provisioning something new.
Confirm The Symptom Start with the pod event:
...
A connected or host-backed virtual CD-ROM can become operational noise during VM maintenance. It can trigger device-lock prompts, confuse vMotion or storage work, and leave operators answering vCenter questions that have nothing to do with the actual change. If the VM is already locked by storage or vMotion tasks, clear the task contention first; see vSphere CSI CNS ExtendVolume Triage.
For long-lived Kubernetes nodes and platform VMs, a CD-ROM is often unnecessary after provisioning. If one remains, it should be either disconnected as a client device or removed intentionally.
...
A vSphere CSI controller log that says a CNS ExtendVolume task is pending does not immediately tell you whether the workload is blocked by resize, attachment, mount, or a stale backend task. The first job is to map the CNS volume ID back to Kubernetes state and avoid turning a storage delay into a destructive rollback.
The useful triage order is:
CNS volume ID -> PV -> PVC -> pod -> VolumeAttachment -> CSI controller logs -> vCenter task state Map The CNS Volume ID Start by mapping the reported CNS volume ID to the Kubernetes PV and PVC:
...
When Kubernetes nodes reboot during a storage incident, the cause matters. An in-guest reboot, a Rancher/system-upgrade action, a human SSH session, and a vSphere HA reset all have different follow-up work.
The useful pattern is to prove the reboot from multiple layers before assigning cause.
Start With Kubernetes Check node readiness, boot ID, kernel, and recent events:
kubectl get nodes -o wide kubectl describe node cp-1 | grep -A5 -E 'Conditions:|Events:' kubectl get events -A --sort-by=.lastTimestamp | grep -i 'reboot\|node' kubectl get node cp-1 -o jsonpath='{.status.nodeInfo.bootID}{"\n"}{.status.nodeInfo.kernelVersion}{"\n"}' Kubernetes can tell you that a node rebooted. It usually cannot tell you why.
...
Replacing Kubernetes nodes from a fresh OS template can be faster than repairing legacy VM drift in place, but speed only helps if the risky work is moved out of the maintenance window without creating identity conflicts.
The useful rehearsal pattern is to pre-create replacement VMs from the current template, leave them powered off, and join them one at a time during the window. That turns the maintenance window into a controlled cluster-change sequence instead of a race to clone, customize, debug, and drain all at once.
...
Longhorn has a CLI, longhornctl, but installing it should not change the operational source of truth during maintenance. For Rancher-managed Longhorn clusters, the most useful day-to-day state still usually comes from the Longhorn UI and Kubernetes CRDs such as volumes.longhorn.io, replicas.longhorn.io, nodes.longhorn.io, engines.longhorn.io, and settings.longhorn.io.
The practical boundary is simple: use longhornctl as an operator tool, not as a reason to skip in-cluster evidence.
Install Locally For an Ubuntu workstation, a user-local install avoids system-wide package changes. Confirm the workstation architecture first:
...
After an RKE2 upgrade or node maintenance event, Calico can fail readiness even when the node itself is Ready.
One failure pattern is node-local and easy to miss: old Calico or Typha processes survive from a previous RKE2 runtime path and keep owning the ports that the current pods need. Kubernetes shows the new pods as unhealthy, but the real conflict is on the host.
This is not a YAML problem first. It is a process ownership problem.
...
Burn-rate alerting is easier to operate when the team understands the concepts before touching Prometheus rules.
The implementation details matter, but the operational idea is simple: alert when a service is consuming its allowed failure budget too quickly. That is different from alerting because a metric crossed a convenient threshold.
This note explains the mental model behind burn-rate alerts. For Prometheus examples and alert rule structure, see SLO Burn-Rate Alerting With Prometheus. For dashboard design after the alert fires, see What To Put On An SLO Dashboard.
...
Certificate automation is not finished when the first certificate is issued.
The operational work is the lifecycle: issuer health, renewal timing, DNS or HTTP challenge reliability, secret ownership, expiration alerts, and safe rotation.
This field note focuses on cert-manager as a Kubernetes platform component, not just as an application dependency.
Define Certificate Ownership Every production certificate should have a clear owner.
Track:
hostname. namespace. owning application or platform team. Certificate resource name. Issuer or ClusterIssuer. challenge type. backing secret. renewal policy. alert route. If a certificate expires and nobody knows who owns it, the platform has an ownership problem, not only a TLS problem.
...