AI Training Gig Work Is as Strange as It Looks
I had my first “AI interview” recently. When I’ve told people this, they usually ask, “Do you mean an interview for an AI company or an interview conducted by AI?” I mean both of these. The company is Mercor, a well-funded contracting firm that seeks experts to help train AI models. They reached out to me because of my software development experience, but they also have tasks including voice acting, financial analysis, and chemical safety.
My interview process began with a 10-minute chat with a human Mercor representative who had very little knowledge about the software engineering contracts I applied to. I suspect that she was mainly talking to me to confirm that I was a real person, and to verify that my face matched the profile picture on my LinkedIn account. From there, I submitted documents and went through a few “assessments”, which took the form of video interviews with a pulsating blob with a feminine voice. My AI interviewer was tirelessly cheerful, responded to any name but didn’t offer any of her own, and answered most of my questions with the kinds of bland generalities you’d get from ChatGPT or Claude. A timer in the corner ticked up to a time limit, but when I reached it, my interviewer was unable to end the call; she prompted me to do so, but when I instead asked questions, even irrelevant questions like “How do you write a ‘hello world’ program in Java?” or “What are some fun activities in San Diego?”, she kept on providing answers.
I’ve accepted a couple of contracts with Mercor and I’ve done some paid onboarding tasks, but very little beyond that; Mercor seems to have onboarded way too many people at once. Mercor’s listed pay rates look good, but there doesn’t seem to be enough work yet for me to consider this even a part-time commitment. I can’t go into too much detail about the work I intend to pick up, but some of it involves evaluating the outputs of experimental AI models to see if, for example, they’re editing source code as competently as a senior software developer would. This is the “human feedback” that is so vital to the reinforcement learning from human feedback technique that makes the output from modern large language models seem so human-like.
There are other companies doing the same type of AI work as Mercor does. I spoke to one nearly a year ago, and my experience onboarding with Mercor seems similar: an aggressive push to get “experts” in the door, followed by many reports online of experts being told to, in so many words, hurry up and wait. I’ve been told to keep watching the company’s internal chat rooms for new opportunities, which could show up at any time, day or night, but it’s hard to stay motivated when I’m not being paid to sit on the bench. I’ve worked jobs before where I’ve had to be on call for potential operational issues, but at least in those cases I’ve been paid for my time even while idle, so I’m more willing to help out when needed.
If there’s a silver lining around this sort of AI company gig work, it’s that I can mix time spent on freelance tasks with my other time commitments, like gym classes, volunteer events, and personal commitments. I’ve repurposed an old phone as my “work phone” and put Mercor-related apps on it, but so far, it’s been pretty quiet. It’s disappointing to see so little work available for the company’s many experts in waiting.