Blake Golliher

Field CTO at VAST Data. I write open-source tools that move live data between storage clusters, and measure it on the way.

See the tools
Contact
Blake Golliher

Moving data

Most of my work is moving data people are still using: buckets, exports, whole clusters. The address stays put. What’s underneath changes.

A snow-covered railroad hump yard seen from above, steam rising over rows of freight cars and fanning switch tracks
Proviso hump yard, Chicago & North Western Railway, December 1942.

Tools I’veauthored

shunt

S3, Go

An S3 proxy that moves a live bucket from one cluster to another while clients keep reading and writing it. Same endpoint, same bucket name, same keys. When the move is done, it steps out of the way.

Proof of concept. It has run on Garage, MinIO and VAST.

shunt on GitHub
      
      
        
        
      
      50%
Animation of shunt moving a bucket between clusters: clients write through shunt to cluster A, the share of writes going to cluster B is ramped up, the mover copies the rest, shunt cuts over, deletes the source copy and steps out so clients talk to cluster B directly. The slider sets the share of writes sent to cluster B.

mongoose

NFS, Rust

Copies everything under one NFS path to another, keeps the copy current while people still use the old server, and cuts over only when every file reads back identical from both.

One static binary. It speaks NFSv3 itself, so nothing gets mounted.

mongoose on GitHub
Animation of mongoose copying an NFS tree from old-server to new-server: it scans and copies, people change files on the old server, sync copies only the changes, and the cutover reads every file back from both servers and compares them before reporting that the trees match.

vamoose

NFS, S3, Rust

The distributed edition of mongoose. Workers on many hosts copy NFS to NFS from a Parquet index and claim shards with conditional writes in S3, so no central service decides who copies what.

An optional coordinator and terminal dashboard add stop, resume and abort.

vamoose on GitHub
Animation of vamoose: vamoose prepare uploads a Parquet index of the source tree to S3, then workers on four hosts claim shards with conditional PUT requests; one wins each shard and the others are refused with 412. When host-c goes down, its claim goes stale and another worker deletes it and claims the shard. Finally vamoose verify compares both trees and writes a metadata certificate.

nfs-walker

NFS, Rust

Walks an NFS export with READDIRPLUS and work-stealing threads, and streams every entry into sharded Parquet. The same binary serves a web dashboard that runs 36 queries over the scan.

810 million files in under 20 minutes, from one host.

nfs-walker on GitHub
Animation of nfs-walker: a directory tree grows as workers list each directory with READDIRPLUS, and every entry found is written to one of 32 Parquet shards. The counters replay the 810.8 million-file production run as a time-lapse. Then nfs-walker serve starts the web dashboard built into the same binary, which pages through charts queried live from the Parquet files.

nfs-gaze

NFS, Rust

Watches NFS client latency one operation at a time, READ, WRITE, GETATTR and the rest, straight from /proc/self/mountstats. No kernel module, no eBPF.

Filter to the ops you care about, snapshot to JSON, export to Prometheus.

nfs-gaze on GitHub
Animation of nfs-gaze watching an NFS mount: IOPS, round-trip time and execute time for each NFS operation, refreshed every second from /proc/self/mountstats. A write latency spike lights up the WRITE row, then the view is filtered to the metadata operations GETATTR, LOOKUP, ACCESS and READDIRPLUS.

s3slower

S3, eBPF, Go

Times every S3 request leaving a Linux host from inside the TLS libraries, with eBPF. It works with any client and needs no SDK wrapper, no proxy and no code change.

Prometheus metrics on port 9000, installed as one RPM.

s3slower on GitHub
Animation of s3slower: requests from boto3, the AWS CLI, mc, rclone and elbencho pass the eBPF probes in the TLS library on their way to the S3 endpoint and back, and each one appears as a row with its latency.

vast-bucket-policy-manager

S3, TUI, Go

A terminal UI for listing, editing and applying S3 bucket policies on VAST, AWS or any S3 store. It checks the JSON and the IAM structure as you type.

Every save and delete writes a before-and-after backup first.

vast-bucket-policy-manager on GitHub
Animation of vast-bucket-policy-manager: a profile is picked, the bucket list loads, and the bucket policy opens in the editor with a typo, which the validator flags. The typo is fixed in INSERT mode, the policy validates, and saving writes a backup before the bucket is changed.

Twenty yearsin infrastructure

Linux user since the late ’90s, closet BSD fan, unabashed storage geek. Off the clock: music, and Batman comics.

  1. VAST Data

    Field CTO

    2025 – now

  2. Tesla

    Sr. Staff Engineer

    • Platform Engineering ownership of the Storage Plank
    • Responsible for Platform Engineering Object Storage company wide
    • First principles around service offering and ownership

    Nov 2021 – 2025

  3. Instacart

    Sr. Engineering Manager

    • Responsible for Incident Response, and RCA
    • Dev/Staging environment Management
    • Observability Engineering
    • As well as other things (Cloud Spend, WebServ layer, etc.).

    Feb 2021 – Nov 2021

  4. Box Inc.

    Senior Manager Engineering

    • Built and developed talented teams of engineers solving problems through software
    • Co-led a hybrid team of engineers modeling finances, helped win massive discounts on public cloud
    • Took over and hired a brand new Observability Engineering team
    • Took over and led a brand new Data Analytics Infra team

    Dec 2018 – Feb 2021

  5. Apple Inc.

    Security Engineer

    • Software Engineering / Delivery Org Hardening
    • Proxy Services DRI
    • Ensuring secure practices in custom software for macOS/iOS build trains

    Dec 2017 – Dec 2018

  6. Amazon AWS

    Sr. Technical Account Manager

    • Led automation efforts inside of AWS Global Support
    • Provided support to AWS's largest customer
    • Founded Field SRE team

    Feb 2017 – Nov 2017

  7. Apple Inc.

    SRE Manager

    • Technical vision and leadership for iTunes Storage Infrastructure
    • Cofounded Blob SRE team working with Development (DevOps)
    • Managed team building a self provisioning web service

    Aug 2011 – Feb 2017

  8. DreamWorks Animation Studios Inc.

    Sr. Architect

    • Designed and built next generation storage and network infrastructure
    • Supported render animation, backend support infrastructure, and Oracle DB
    • Designed and implemented first large scale SAN in the environment

    Aug 2010 – July 2011

  9. Facebook Inc.

    Sr. Architect

    • Owned production storage for photo, video, log, and everything in between
    • Hired staff of 4, built run books, automation and monitoring

    Nov 2006 – Nov 2008

  10. Yahoo! Inc.

    Architect

    • Responsible for architecture and engineering storage for Yahoo! Mail, Oracle DB and a few other properties
    • Built run books, started automation and monitoring of devices

    Sept 2005 – Nov 2006

Education: Mount Sierra College, Pasadena, CA. Major, Telecom Technologies. Minor, Goofing Around.

Write about storage, migrations, or world domination.

Grand Rapids, Michigan

GitHub