A-Google-Cloud-exec-shares-the-two-ways-she-evaluates.jpeg

A Google Cloud exec shares the two ways she evaluates creativity in job interviews

Google executive Yasmeen Ahmad is looking for something specific when hiring engineers — and it’s not just technical know-how.

Ahmad told Business Insider that the typical software engineering interview used to focus on detailed coding tests and test suites. Now, as she hires for a forward-deployed engineering team, which will work with customers, she said she’s prioritizing people with fresh ideas.

The strongest candidates are “able to think outside the box,” Ahmad, director of Google Cloud’s data cloud, said. “They’re able to think outside the frame of how we would have normally described a problem.”

The executive added that candidates who take a traditional approach to engineering aren’t performing as well in her team’s interviews. The ideal candidate nowadays, she said, can demonstrate creative problem-solving by using AI to reimagine traditional processes. She said she evaluates that type of thinking in two ways:

1. Constant experimentation

Ahmad said she looks for candidates who are constantly “tinkering” with new tools. That gives her an immediate signal that they’re creative thinkers.

“When you’re interviewing them, they’re naturally immediately talking about, ‘oh, last week I had tried AI in this context, and this is how it made me better at doing my job in this way,'” Ahmad said.

These candidates aren’t trying new tools because their boss told them to or because it’s the new cool thing to try, she said.

“They’re the early adopters,” Ahmad said.

Tech executives have told Business Insider that side projects are becoming increasingly common for candidates to demonstrate their aptitude in interviews. However, Ahmad said candidates don’t need to have a GitHub repository of projects they’ve worked on in their spare time.

“It doesn’t have to be pet side projects, because people are busy,” Ahmad said, adding that workers can experiment on the job by trying out new ways to speed up their work.

2. Scenario testing

AI is being used more often throughout the interview process — in some cases, illicitly by job seekers, and in others, as a way for employers to test candidates’ AI capabilities. As these tools reshape hiring, Ahmad said scenario-based testing has become central component to the interview process, giving hiring managers a better way to assess creativity.

Ahmad said she’ll ask candidates how they would approach a scenario involving AI tools in an industry where they have no domain knowledge.

For example, if the example related to healthcare, a traditional candidate might say that they would take all the patients’ unstructured PDFs, feed them into a single LLM prompt, and ask it to generate a summary for the doctor. That would be a “massive liability,” Ahmad said, because in that scenario the candidate assumes AI can inherently understand the timeline of events or clinical context of an image by looking at it.

Ahmad said she’s looking for a candidate who can “find solutions in a way that breaks the chains of how that workflow process has traditionally gone.” So someone might suggest building the semantic context for the imaging data before the model sees it. Next, they would build a specific framework to ensure the agent is operating in the right time frame of data. Then, they would recommend designing a multi-step process that includes a continuous evaluation loop.

“We aren’t just hiring people to write prompts,” Ahmad said. “We are hiring people who can foresee how a model might silently fail in a high-stakes environment, and who know how to build the automated evaluation loops to catch it before it does.”

She said asking these sorts of questions to vet creativity is especially useful as AI transforms the software engineering industry by automating core parts of the job.

“We’re seeing the human role is evolving to more of an orchestrated role,” Ahmad said. “So rather than having to write all of the detailed code, it’s ‘how do I actually express my intent to a multi-agent system now and have that multi-agent system execute on that intent?'”




Source link

More-hiring-managers-want-you-to-prove-youre-good-with.jpeg

More hiring managers want you to prove you’re good with AI during job interviews

Leaders at the software company Canva used to wonder whether job candidates were secretly using AI during technical interviews.

By early last year, that concern gave way to a bigger question: How good are they with AI?

Managers saw the company’s engineers getting more done with the technology, so they needed to ensure new hires could do the same.

“We just flipped the script and went, ‘OK, we’re going to invite you to use AI,'” Brendan Humphreys, Canva’s chief technology officer, told Business Insider.

The result, he said, has been stronger hires better equipped to wield powerful AI tools to help write code and solve problems.

Canva is one of a growing number of companies — including Meta and McKinsey — that are inviting some job candidates to use AI in parts of the hiring process.

Broadly, when ChatGPT emerged in late 2022, many employers worried that job seekers would use AI to help talk their way past interviewers. Yet as the technology becomes more capable and embedded in daily work, a number of companies are moving from policing it to evaluating candidates’ AI know-how.

That’s what happened at Arcade, an IT infrastructure startup. The company has always asked technical candidates to complete a take-home exercise. Yet now, it expects them to use AI in the process, Alex Salazar, the company’s cofounder and CEO, told Business Insider.

As the technology’s capabilities surged over the past year or so, he realized that candidates would likely turn to AI regardless of whether Arcade sanctioned it. Ultimately, Salazar said, the company wants its workers, including new hires, to use AI.

“So why are we creating this artificial test that doesn’t even really reflect the work they’re going to do when they get here?” he said.

Humphreys came to a similar conclusion at Canva. To factor in AI, he said, the company reworked its technical interview to make the questions “complex, ambiguous, and problematic.”

“If you just dump the question that we’re giving you into an AI, you’re going to get a substandard answer,” Humphreys said.

To land a job at the company, which has about 265 million monthly users of its graphic design software, technical candidates need to know how to thoughtfully question AI, he said.

Show us you can work with AI

One way to avoid concerns that candidates might be leaning too hard on AI is to have job seekers show their work. In Canva’s case, the company asks candidates to share their screen during a technical interview.

“We want to see the interactions with the AI as much as the output of the tool,” Humphreys said.


Brendan Humphreys

Brendan Humphreys, CTO at Canva

Courtesy of Canva



Arcade tells candidates to use whatever AI tools they want on their exercise, then include a transcript of their conversations with the AI. The idea is to learn who knows how to do the job and to work with an agent. Doing so, Salazar said, comes with a “very real learning curve.”

He said that the shift to allowing AI use in the exercise meant that Arcade placed greater emphasis on a candidate’s “taste.” That sensibility is important, he said, because AI can kick out answers, yet the best results often come from repeated iteration with these tools, he said.

“It’s going to show their ability to use the AI, but it’s also going to show what they think ‘good’ is,” Salazar said of candidates’ interactions with AI.

‘Ride the dragon’

Other companies want workers to demonstrate their AI acumen during the hiring process, too.

In a June post on an internal message board, Meta said it was developing a coding interview in which candidates could use an AI assistant, Business Insider previously reported.

That mode of working, Meta wrote, was “more representative” of the environment in which future developers would be operating. It also makes “LLM-based cheating less effective,” the company said, referring to large language models.

The consulting firm McKinsey & Company is piloting a change to its graduate recruiting process, asking candidates to use the company’s internal AI assistant, Lilli, during case interviews to assess how they work with the technology, several media outlets reported in January.

The acceptance of, or even the preference for, AI in some parts of hiring doesn’t mean companies will welcome job seekers who use the tools to misrepresent their skills. Even if a candidate gets away with it at first, hiring managers are likely to eventually discover that someone doesn’t have the goods, Susan Peppercorn, an executive coach, told Business Insider.

That’s because candidates who complete an assessment, for example, “are going to have to explain how they arrived at their thinking,” she said.

Understanding that thought process is what Canva seeks in its hiring, said Humphreys, who oversees roughly 2,600 technical employees in roles including software engineering, IT, and machine learning.

It’s a way of seeing whether a candidate makes sound technical decisions when it starts producing code, he said.

“What we’re testing for now in our interview process is an ability to harness that power, to control that power — to kind of ride the dragon,” Humphreys said.

Do you have a story to share about your career? Contact this reporter at tparadis@businessinsider.com




Source link