Silicon Valley is an area with extremely intense competition in almost every aspect. Technology companies, both large and small, strive to attract talented employees with a key tool being "compensation." This compensation keeps increasing, including salaries, bonuses, and company shares. Recently, another factor has been added to the compensation equation, which is "AI Inference Compute." Or "AI processing costs." This comes after the enormous rise in these costs at many companies worldwide.
As generative AI tools become undeniably crucial components in software development, the cost of running these AI models, known as "Inference," has become a significant expense that many companies can no longer overlook. Moreover, tech companies compete to access GPUs, essential resources for running AI, which are usually carefully allocated to the most important projects.
Recently, a new trend has emerged as tech job applicants have started asking, “If I get the job, how much AI budget will I receive?”
Thibault Sottiaux, chief engineer of Codex (OpenAI’s coding assistant tool), posted on X, “I’m increasingly hearing this question during interviews: how much Inference Compute will candidates get to create work with Codex?” He added that AI usage per person is growing faster than the overall user base, reflecting that AI Compute resources are becoming scarcer and more valuable.
This shortage is changing engineers’ perspectives on “work” and “compensation.” Greg Brockman, president of OpenAI, has candidly stated, “The amount of Inference Compute you can access will increasingly determine overall software performance.”
Simply put, without sufficient computing power, we may produce less work than others, which could directly affect our career paths. Thus, in the AI era, “access to AI” may be as important as a high salary or large stock options.
Additionally, the work environment at tech companies is evolving, with AI access or AI subscription payments becoming standard employee benefits.
AI Tokens are a fundamental part of the generative AI world. AI models convert prompts or data into Tokens, which are numerical units processed to generate responses.
One Token can vary in size; for example, the word "Hello" might be split into "Hel" + "lo" and converted into a number sequence like [102, 504]. Importantly, Tokens are also the units used to calculate AI service fees.
Peter Gostev, head of AI at the startup Arena, suggested that companies like OpenAI and Anthropic should create job platforms where firms can post openings specifying compensation in both salary and AI Tokens for those positions.
Furthermore, technology companies are seriously reconsidering this approach as the new AI usage costs can be substantial and must be closely monitored like employee salaries, since employees’ AI usage is becoming a direct company expense.
According to Levels.fyi, the salary of a software engineer is about $375,000 per year. Adding $100,000 per year for Inference costs raises the total cost per employee to $475,000 annually. In other words, in the future, AI costs could account for over 20% of employee expenses.
The key question is, "Does the money spent on AI yield a worthwhile return?" In the cloud world, efficiency is measured by profit per GPU hour, but for employees, it might be measured by output generated per dollar spent on Inference.
Tomasz Tunguz from Theory Ventures said he experiments with AI daily and can automate up to 31 tasks per day with Inference costs of just $12,000 per year. Therefore, if an engineer’s AI cost reaches $100,000 annually, they should be producing eight times more work.
Source:Business Insider
Follow the Facebook page: Thairath Money at this link -https://www.facebook.com/ThairathMoney