Aaron's Blog

Tech, tinkering, and occasionally, a banjo tune

My Workspace

This post is the last in the Support Driven Writing Challenge series that I've been participating in lately, albeit a bit late. This post is about my workspace.

As a fellow support driven community member and remote worker Chelsea noted, being remote means that I can work from anywhere. In my case, it's mostly true (given that I work third shift). My space has been a place that I intentionally crafted to be a place that would make going from a first shift job to a third shift one tolerable, if not at least somewhat enjoyable.

desk

That's the hoss of a desk that I hand-built. It's iron pipe and stained pine, and all of 300 lbs (trust me, I attempted to lift it myself). I had previously used a Standesk 2200 while I was at Rackspace, but found the prospects of standing all night to be a bit...unpleasant.

I wanted my desk to be something solid, rugged, and something that would hold up to not only normal work, but something that would also function as sort of a work bench when I tinker with electronics. I also wanted it to be something that I could work at all night and not grow too terribly bored at, which is why I added the little terrarium you see in the picture.

The other elements of the desk are as follows:

Overall, I feel that the setup has worked well for working on third shift when I'm not doing work at Starbucks/other coffee shops. The only downside will be when the occassion next strikes us to move...I think I'll use movers for the desk.

It Came From the Night Shift

Whew, I'm catching up on posting for the Support Driven Writing Challenge, and I'm fairly caught up at this point. So this week's post is a 'Day In the Life' of whatever it is you do. Since I'm in my first ever third shift position, this should be interesting!

The Schedule

Before coming to DigitalOcean, I was in a 'normal' 9-5 position. Then, I flipped completely over to 3rd shift when I joined DO. Now I'm working 11pm CDT to 7am CDT. I'll walk you through what my typical day looks like:

  • 14:30-15:00: Wake Up, get ready
  • 15:00-17:00: Head to Starbucks to write, learn, catch up on stuff while people are in the office, maybe run some errands
  • 17:00-17:30: Head home
  • 17:30-20:00: Start cooking dinner, run other errands
  • 20:00-22:00: Take a nap and get ready for my shift
  • 23:00-07:00: Do all the work

That's a quick breakdown of the schedule. But what, you ask, does a Customer Success Engineer actually do?

What it is I say I do at DO

Let me start by explaining my team's mission: [To] forge and foster close relationships with key customers by ensuring their success on our platform. What does that mean practically? Well, there are really three 'hats' that our duties fall into:

  • Account Management
  • Tier 2 Support
  • Solutions engineering/consulting

So on any given night, I can have a mix of tasks that is a random collection of those duties. Right now, there is a pretty heavy leaning to support and account management requests since I work primarily with our APAC customers. Let me give you an idea of how these sorts of things fall into a normal night.

I'll usually come in and give a nod to my colleagues Huck and Jon (depending on the night) on Slack as they're headed out for the evening. I'll also ping our Support team and Cloud Operations team to let them know that I'm in, and to ping me if they need anything. From there, I'll catch up on what's gone on during the day. This usually consists of reviewing what's happened in our Slack channels (we have a general channel, as well as a 'standup' channel for any notable issues that occurred, or that need follow up) and perusing my email. A quick note here: I get TONS of email. I've got everything filtered so that anything that doesn't come in through HelpScout, and isn't a general notification gets pushed to the top of my inbox.

After I've caught up on the day and email, I'll check and see if there's been any movement on any Jira tickets I've opened previously, or if anything is needed from me on them. Nine times out of ten, one of my DayWalker colleagues have addressed the Jira if it's a customer-facing/impacting issue.

From there, the rest of my time is spent responding to support requests and Hatch(our initiative to support startups as they launch) applications. The requests that we get are pretty varied. Most of the time, the requests that come in are that a droplet (our term for a VPS) has become unresponsive, or that the customer can't log into the droplet and need us to boot it into a recovery mode. However, we do get some interesting issues during the night. For example, a customer's MongoDB cluster had an issue that resulted in their metadata getting corrupted, and we had to troubleshoot that. Keep in mind that DigitalOcean is a self-managed platform, which means we don't log into customer droplets. This can make troubleshooting a challenge. It's forced me to get better at thinking through an issue, how I would address it/troubleshoot it, and explaining that process to a customer. We also see issues end up being symptomatic of something larger, and may require the customer to re-architect their infrastructure.

In addition to the customer-initiated support requests, there are also the inevitable issues that arise when dealing with technical gremlins. These tend to manifest themselves in us having to reboot hypervisors (underlying infrastructure that runs customer droplets), and can be for any number of reasons.

Our solution engineering/consulting process is in the process of changing. When I started, each of the Customer Success Engineers at DO were responsible for consulting/engineering requests and calls that came in to our team. Now, our Customer Success team does more of a qualification process to see if a customer would need to have a chat with our Solutions team, who are the ones who take on more of the engineering/architecting sort of work. When I have those sorts of calls scheduled at night, they now tend to fall into the pre-qualification sort of vein where I chat with the customer to see what problem they are currently experiencing, how they have their infrastructure set up, and what their end goal of the re-architecture process is.

When I don't have any requests, or issues to address, it can be a bit lonely/quiet, especially working from home. So I'll fill my time learning what I can via Linux Academy, Udemy, and my ever-growing stack of Oreilly ebooks. When that fails, I'll get up and try and give myself a change of scenery. At 3-4am, this is a bit difficult. I'm not too keen on wandering around in my Cthulu slippers and having our local law enforcement called due to the 'creeper in octopus shoes walking around the neighborhood' (note: this has NOT happened yet, and I aim to keep it that way). So sometimes, I'll go outside and smoke my pipe, or at the kitchen table, or in front of the TV with some anime playing in the background. In the case of last night, a fire at 2am sounded grand, so I made one and worked outside last night.

workFire

I wrap up my shift by doing a brief handoff with our first shift CSE's, greet the wife as she wakes up and heads out to work, and then promptly conk out for the 'night'. I'll be writing a follow up post to this about surviving 3rd shift, since I feel like I could write a small book on it. Stay tuned for more!

Cheers,

Aaron

The Support Stack At Digital Ocean

It's week four of the Support Driven Writing Challenge, and sadly, I'm a bit behind what with the US holiday and the customary food coma. That said, let's get cracking.

The Ubiquitous Ticket Queue

I'll start off by saying that the most common way of supporting our customers at DigitalOcean (from a Customer Success Team standpoint), is via support ticket through our home-grown system. That's where most of our work comes from, be it customer-initiated tickets, or automated tickets. A majority of the time, tickets work decently. But there are some pitfalls, especially when working for a self-managed platform like DigitalOcean. Namely, that the ticket approach can seem impersonal

Email Support

We also use email to support customers working directly with our Customer Success team. This is done via Helpscout, and works fairly well. Emails, while they don't have an SLA attached to them, typically get responded to quite quickly. Emails also tend to not deal so much with in-depth technical issues, and when they do, we usually push that sort of thing to a ticket, so that everything is documented and auditable.

The Not-So-Common Phone Call

At my previous two positions, phone calls were the bread an butter of what I did, especially at Rackspace. I typically was on the phone with a customer walking them through and issue, and troubleshooting an issue on their server while they watched. It was a bit nerve-wracking at times. However, since DigitalOcean doesn't have a support line, the phone support that we typically do tends to be on the more consultative side. For example, my team works with quite a lot of startups who come through our "Hatch" program. Many of the calls that we do are introductory, and I like to think of them as a first date. You know, the "Hi, who are you? What do you do? Here's what I do" sorts of calls. On the rare occasion, we'll hop on the phone with a customer and troubleshoot over the phone.

A View of the Other Side of the House

Just to give an idea of what our front lines teams do, they typically operate off of tickets, but also include social support via Twitter or Facebook. Our support team definitely sees a higher volume of tickets than our Success team, so our ticketing system shines when they're able to address quite a few tickets in a relatively short amount of time.

Some Thoughts

I wasn't around for the choosing of any of the components of our support stack, and it's been interesting to read up from other members of the Support Driven community who have had that opportunity to build theirs out. That said, I've got no opinions really on the platform that's being used to provide support (a la Zendesk, Kayako, Intercom, etc.).

I will say that I have a bit of a love/hate relationship with ticketing systems. For all the benefits they provide, I find ticketing systems to be the least personal out of any support tool. That may be the communication researcher in me speaking, but I think the loss of verbal/nonverbal queues can exacerbate situations in which those sorts of things are necessary. I can tell a customer in a reply that I empathize with them, and that yeah, it's terrible that X happened, but they don't get to see that. Instead, they're forced to infer my tone/meaning in my reply. And no matter how much I try to instill empathy into a response, there will always be a case where that falls short.

Phone support, if done well, can eliminate some of that--namely, the lack of verbal queues. This all goes back to training, though. If a support agent isn't empathetic, and doesn't take the time to put themselves in their customers' shoes, all the phone support in the world won't do any good (I'm looking at you, Comcast & Time Warner). So I think (and perhaps I'll do a post on this in the future) that while a support stack is important, and should be chosen carefully, it's really a collection of tools to accomplish one purpose: supporting a customer. If the tools don't do it well, then they're not worth choosing. Likewise, it's important to have people on your team who know how to use those tools effectively. If you choose a tool that doesn't allow for being able to pick up on verbal/nonverbal queues, the person behind that tool has to know that weakness, and compensate for it.

Well, I've rambled on long enough, and I've got to get cracking on the next post! Stay tuned for a Day in the Life Of a 3rd shift Customer Success Engineer at DigitalOcean.

Wildwood Flower

at down for a bit with a pipe and my banjo. Here's what I ended up recording:

My Story

I've told pieces of my story in recent posts. You know, the ones about Homebrewing being a salvation, and moving to a role in Customer Success at DigitalOcean. I don't think I've ever posted about the long, circuitous route that my career has taken. Perhaps it's time to tell you the story. So sit back and grab a cup of joe...this ride is known to cause whiplash. Trust me, the trail from aspiring academic to customer success engineer is not exactly the smoothest.

The Younger (Wonder) Years

My single-prop Cessna carrying supplies to a remote African village hit a patch of turbulence, sending my stomach lurching...or at least that was the dream I had when I was a kid. All I wanted to do was be a pilot. I was going to go to school like all good kids do, graduate, and be a bush pilot. Things changed when I realized that the school I wanted to attend was planning on closing.

In the mean time, I had taken a trip where we traced the 101st Airborne's route through Europe during World War II. It was the trip that forever kicked off a desire to learn foreign languages (specifically German). I returned from the trip in the Summer of 2002, met with my guidance counselor, and had her completely rearrange my sophomore class schedule just so that I could learn German (I had also been taking Spanish up to that point). Over the next three years, I did everything I could possible do to learn German. As a sophomore in high school, I'd aced the county German exam (and found 3 grammatical mistakes), taken the national German exam (scored a 96), and was invited to participate in the American Association of German Teachers Summer Study Program. Not to mention, I was the first person from TN in 12 years to be selected for that program.

Until this point, I had thought that I was going to spend the rest of my life and career in Germany, with a German wife, happily speaking German. All roads led to Rome...er, Berlin, in this case.

College (Or, The Real World Sets In)

I graduated in 2005 having kicked the proverbial pants off of my high school's German program. Along with two friends, we'd forced the school to start providing German AP classes, had all scored high marks on the German AP exam, and were going to continue our German studies in college. In fact, I'd waived up to senior level courses, completed the required classes, done another exchange stent in Leipzig during the 2006 Weltmeisterschaft (World Cup), and was merrily on my way to being a career academic. All the while, I'd somehow acquired a bit of technical acumen (designed a website for a local non-profit, fixed computers for friends, etc.)

Here's where the path gets a bit wonky. While I'd completed the German degree requirements, I was at a four-year institution that required me to complete the whole four-year stent in order to earn my degree. I was two years in, and had no idea what I was going to do. Me being the chatty dude that I am, I picked up another degree in Speech Communication. What better combination? I could ramble in German, and put a bit of communication theory behind it. "How to Be Persuasive in German" should have been my senior project...

Oh, did I mention I met an amazing woman along the way? We knew each other since high school, but started dating in college. Our relationship had blossomed to the point that the thought of pursuing further German studies was going to take me far away from her. We'd spent 2006-2009 dating long distance, and I wasn't keen to be further away from her (we also began our relationship by starting to date each other two weeks before the three-month exchange trip I took to Leipzig).

I decided then that I would pursue further studies in Communication, and enrolled in the MS of Communication and Information program at the University of Tennessee.

Communication is Cheap...Literally

After Ashley and I graduated, we both ended up back in Tennessee. I had started my Master's program, and she was studying to be a forensic pathologist. While searching for a job on campus that didn't involve spending my days at the local Chick-fil-A breading chicken, I started applying for assistantships. My department's assistantship paid ~$300/month (which was NOT going to cover rent), so I looked to other departments. After 13 rejections, I landed a job at the Office of Information Technology at the helpdesk.

Over the next two years, I worked on just about everything related to desktop support and have the horror stories to prove it. Professor-dad screaming about how his non-student daughter's desktop got infected by malware and we're not helping him? Check. Crazy cat lady's ancient, dust-bunny infested desktop that has barely enough resources to run notepad? Check. The student who has a completely disarticulated hard drive and wants us to 'fix it?' Oh yeah, check.

Graduation was looming, as was my marriage to Ashley. I decided that a $1000/month job wasn't really going to cut it for a young family (and neither was surviving on the rest of my student loans), so I met with my then manager, beefed up a resume with her help, and eventually ended up with a contract job at Pilot/Flying J where I re-architected their desktop deployment process as contractor in their QA lab. It's safe to say that the deployment process was...archaic. In the first two months of my contract, I was able to reduce the number of 'golden images' from ~30 to two, decrease the average deployment time from two weeks to four hours, and decrease the total space taken up by the images from ~100gb to < 16gb. At the end of the contract, I moved next door to a small MSP--I wasn't going to be able to get them to use WSUS (which would have brought its own challenges) and was still going to be a contractor with no benefits.

Starting Support, Leaving the Southeast

The move to the MSP I worked at was my introduction to a proper ticket queue (often hundreds of tickets). I started beefing up my support chops there, where I supported everything from random line-of-business applications for doctors, to Outlook, Exchange, MSSQL, and everything in between. You read correctly--I was a Winows admin at one time (shudders). But honestly, without that job (or really any of my technical positions), I wouldn't be where I am. It was there that I also beefed up my technical chops. I took the little that I knew from my job at the University of Tennessee's helpdesk, and the bit that I knew about Windows and applied that in this job.

Eventually, I realized that if I wanted to grow technically, it wasn't going to be as a Windows admin in the Southeast US. I'd come to find out that if you're not in Atlanta, Georgia, and not in/around the research triangle in North Carolina, finding a technical job that pays well and wasn't one of those 'purple squirrel' type positions was nigh impossible. If I was going to get anywhere, I needed a mentor. Thankfully, one of my best friends took me under his wing. For around 6 months, we met every Tuesday night at a local coffee shop talking Linux, operations, and everything in between. He even gave me a project that was designed to incrementally teach me more about the basics of being a Linux admin.

At around the six-month mark of our coffee shop meetings, I started applying to companies outside of Tennessee. Usenix, AWS, NIST, and Rackspace were the companies on my list, and amazingly, I got interviews with each one. The mentorship had paid off, even if I was a bit green and hadn't worked on any Linux production infrastructure up to this point. In the end, I ended up getting turned down from AWS, NIST moved too slow, Usenix didn't have a competitive offer, and while I had been turned down from Rackspace initially, they offered a position (Linux Technician) that would allow me to learn, work on customer production infrastructure, and provide the space for me to grow and learn more about being a Linux admin.

Texas and Beyond

Loaded Uhaul
The move from Tennessee to Texas was the hardest thing I've done in my adult life. My wife and I moved 16 hours from all we'd ever known, to a town where we knew two people (my aunt and uncle, who have been our surrogate parents in the time we've been in San Antonio). The drive was hard, and the first 6 months of being in Texas almost broke us. We missed the mountains, the people, and our homes. Ultimately, we found a home at Rackspace. Some of our dearest friends were made there as we met for the first time at the Tamale Festival in sub 40F weather (unusual for San Antonio).

The time I spent at Rackspace grew me immensely. I went from knowing only what I had learned in the project my mentor had provided me, to working on projects that affected a number of customers, writing automation to take some of the pain out of support as well as reduce the number of hours we spent supporting customers, and mentoring other greenhorn admins.

While Rackspace was an amazing opportunity, I realized that the need I have to keep growing and learning wasn't getting satisfied, and I was getting burnt out staying in the same place. I then started looking for a new role. I needed something that would allow a bit more creativity, as well as further growth. In March, I took a role with DigitalOcean, which you can read more about here. It's been an amazing chance to grow technically, and touch more cutting edge technology than I would have at Rackspace.

Looking back, I can say that each new role and phase of my career led to the other. The path to support (yes, Customer Success is support, at least at DigitalOcean) hasn't been a straight one at all. It's been a bit caddywhompus at times, and has entailed making decisions that I never thought I'd have to make (like moving). But I can see that the roles built upon each other, whether it be developing new technical skills, or improving the ways in which I communicate and relate to customers. I also see the direct and tangible results that being in a mentoring relationship has had on my career. I'll leave with a couple of thoughts: 1.) Unless you know 100% for a fact that support is the career you want to have, starting a career in support is never a straight path. Those of us in the Support Driven community know this, and come from various and sundry backgrounds that have led to where we are. 2.) If you're starting out in a support role, I can't stress enough how important it is to be in a mentoring relationship. It's the difference in say, using two sticks to start a fire, and pouring some jet fuel on that fire. It made all the difference for me, and is something I actively try to pass on as I mentor other admins.