You have to make a lot of decisions when you retire, and among the biggest is what to do with your workplace retirement savings. No matter how much money you have or how you intend to invest it, you have to first choose where your nest egg will live.
You have four basic choices.
Remain in your employer’s plan and just let the money grow until you have to start taking the required minimum distributions (RMDs).
Remain in your employer’s plan while taking installment payments.
Roll over the assets to an IRA at an institution of your choosing.
Take the account balance in cash and pay tax on the distribution to either spend it or roll it into a Roth IRA.
The good news, according to recent research from Vanguard, is that most people faced with this decision over 10 years, from 2011 to 2021, were able to preserve their retirement dollars. Seven out of 10 kept their assets in a tax-deferred environment, and 90% of the money stayed invested, and presumably, grew a bit. Average balances ranged from $239,300 to $418,900.
“More and more investors are on the right road to having a good experience with accumulations. We’re seeing improvements,” says Matt Brancato, chief client officer for Vanguard Institutional.
But, Brancato adds, “the average doesn’t tell you about individual experience.”
And for that, you have to look at some of the less good news, which is that Vanguard found that 30% cashed out their savings at age 60 or later, most with smaller balances. The average amount of these accounts was $39,700. Some had likely simply saved less, and some had been with the company plan for a short time, so had not accumulated a large amount.
The peril of cashing out
Cashing out a small balance might seem inconsequential to you at the time. The account could be one of many that you have, and the tax burden might not seem too much for you to bear. Or you could be intending to pay the income tax due on the distribution and roll the money into a Roth IRA in a conversion. Or the cash might be enticing – and then it’s gone.
“First of all, ‘small’ is a relative term,” says Brancato. “The dollar amount has to be proportionate to the intent. It’s a highly individualized decision.”
One important step if you’re thinking of cashing out is to consider how the amount involved could possibly grow over time and add to your retirement income later on. If your balance is $39,700 now and you think that isn’t much, it could be $78,000 in 10 years, if it grows at 7%.
At Ascensus, another large retirement plan administrator, they display those numbers to people when they initiate a decision that would impact their retirement savings, like reducing their 401(k) contribution. “We serve up a very quick estimate to connect the dots between what seems like a small amount to a much larger amount of money you’d forgo in retirement as a result,” says David Musto, CEO of Ascensus. After seeing that information, “30% of people ultimately choose not to reduce 401(k),” he adds.
That same kind of information may also help people make a decision between staying in their workplace plan after retiring or moving the money to a rollover IRA. While most eventually move money over to their own account within five years, Vanguard’s study shows that the numbers are shifting up for those staying in their workplace plan even after they retire.
Brancato sees the driver of that being flexible plan design, advice and financial-wellness tools that may be part of an employer package. If you want to tap into your money before you have to take RMDs, for instance, your plan would have to allow it, and Vanguard notes that the number of plans offering this nearly doubled in the past five years.
Got a question about the mechanics of investing, how it fits into your overall financial plan and what strategies can help you make the most out of your money? You can write me at email@example.com.
Nvidia CEO Jensen Huang speaks during a press conference at The MGM during CES 2018 in Las Vegas on January 7, 2018.
Mandel Ngan | AFP | Getty Images
Software that can write passages of text or draw pictures that look like a human created them has kicked off a gold rush in the technology industry.
Companies like Microsoft and Google are fighting to integrate cutting-edge AI into their search engines, as billion-dollar competitors such as OpenAI and Stable Diffusion race ahead and release their software to the public.
Powering many of these applications is a roughly $10,000 chip that’s become one of the most critical tools in the artificial intelligence industry: The Nvidia A100.
The A100 has become the “workhorse” for artificial intelligence professionals at the moment, said Nathan Benaich, an investor who publishes a newsletter and report covering the AI industry, including a partial list of supercomputers using A100s. Nvidia takes 95% of the market for graphics processors that can be used for machine learning, according to New Street Research.
The A100 is ideally suited for the kind of machine learning models that power tools like ChatGPT, Bing AI, or Stable Diffusion. It’s able to perform many simple calculations simultaneously, which is important for training and using neural network models.
The technology behind the A100 was initially used to render sophisticated 3D graphics in games. It’s often called a graphics processor, or GPU, but these days Nvidia’s A100 is configured and targeted at machine learning tasks and runs in data centers, not inside glowing gaming PCs.
Big companies or startups working on software like chatbots and image generators require hundreds or thousands of Nvidia’s chips, and either purchase them on their own or secure access to the computers from a cloud provider.
Read more about tech and crypto from CNBC Pro
Hundreds of GPUsare required to train artificial intelligence models, like large language models. The chips need to be powerful enough to crunch terabytes of data quickly to recognize patterns. After that, GPUs like the A100 are also needed for “inference,” or using the model to generate text, make predictions, or identify objects inside photos.
This means that AI companies need access to a lot of A100s. Some entrepreneurs in the space even see the number of A100s they have access to as a sign of progress.
“A year ago we had 32 A100s,” Stability AI CEO Emad Mostaque wrote on Twitter in January. “Dream big and stack moar GPUs kids. Brrr.” Stability AI is the company that helped develop Stable Diffusion, an image generator that drew attention last fall, and reportedly has a valuation of over $1 billion.
Now, Stability AI has access to over 5,400 A100 GPUs, according to one estimate from the State of AI report, which charts and tracks which companies and universities have the largest collection of A100 GPUs — although it doesn’t include cloud providers, which don’t publish their numbers publicly.
Nvidia shares are up 65% so far in 2023, outpacing the S&P 500 and other semiconductor stocks alike.
Nvidia CEO Jensen Huang couldn’t stop talking about AI on a call with analysts on Wednesday, suggesting that the recent boom in artificial intelligence is at the center of the company’s strategy.
“The activity around the AI infrastructure that we built, and the activity around inferencing using Hopper and Ampere to influence large language models has just gone through the roof in the last 60 days,” Huang said. “There’s no question that whatever our views are of this year as we enter the year has been fairly dramatically changed as a result of the last 60, 90 days.”
Ampere is Nvidia’s code name for the A100 generation of chips. Hopper is the code name for the new generation, including H100, which recently started shipping.
More computers needed
Nvidia A100 processor
Compared to other kinds of software, like serving a webpage, which uses processing power occasionally in bursts for microseconds, machine learning tasks can take up the whole computer’s processing power, sometimes for hours or days.
This means companies that find themselves with a hit AI product often need to acquire more GPUs to handle peak periods or improve their models.
These GPUs aren’t cheap. In addition to a single A100 on a card that can be slotted into an existing server, many data centers use a system that includes eight A100 GPUs working together.
This system, Nvidia’s DGX A100, has a suggested price of nearly $200,000, although it comes with the chips needed. On Wednesday, Nvidia said it would sell cloud access to DGX systems directly, which will likely reduce the entry cost for tinkerers and researchers.
It’s easy to see how the cost of A100s can add up.
For example, an estimate from New Street Research found that the OpenAI-based ChatGPT model inside Bing’s search could require 8 GPUs to deliver a response to a question in less than one second.
At that rate, Microsoft would need over 20,000 8-GPU servers just to deploy the model in Bing to everyone, suggesting Microsoft’s feature could cost $4 billion in infrastructure spending.
“If you’re from Microsoft, and you want to scale that, at the scale of Bing, that’s maybe $4 billion. If you want to scale at the scale of Google, which serves 8 or 9 billion queries every day, you actually need to spend $80 billion on DGXs.” said Antoine Chkaiban, a technology analyst at New Street Research. “The numbers we came up with are huge. But they’re simply the reflection of the fact that every single user taking to such a large language model requires a massive supercomputer while they’re using it.”
The latest version of Stable Diffusion, an image generator, was trained on 256 A100 GPUs, or 32 machines with 8 A100s each, according to information online posted by Stability AI, totaling 200,000 compute hours.
At the market price, training the model alone cost $600,000, Stability AI CEO Mostaque said on Twitter, suggesting in a tweet exchange the price was unusually inexpensive compared to rivals. That doesn’t count the cost of “inference,” or deploying the model.
Huang, Nvidia’s CEO, said in an interview with CNBC’s Katie Tarasov that the company’s products are actually inexpensive for the amount of computation that these kinds of models need.
“We took what otherwise would be a $1 billion data center running CPUs, and we shrunk it down into a data center of $100 million,” Huang said. “Now, $100 million, when you put that in the cloud and shared by 100 companies, is almost nothing.”
Huang said that Nvidia’s GPUs allow startups to train models for a much lower cost than if they used a traditional computer processor.
“Now you could build something like a large language model, like a GPT, for something like $10, $20 million,” Huang said. “That’s really, really affordable.”
Nvidia isn’t the only company making GPUs for artificial intelligence uses. AMD and Intel have competing graphics processors, and big cloud companies like Google and Amazon are developing and deploying their own chips specially designed for AI workloads.
Still, “AI hardware remains strongly consolidated to NVIDIA,” according to the State of AI compute report. As of December, more than 21,000 open-source AI papers said they used Nvidia chips.
Most researchersincluded in the State of AI Compute Index used the V100, Nvidia’s chip that came out in 2017, but A100 grew fast in 2022 to be the third-most used Nvidia chip, just behind a $1500-or-less consumer graphics chip originally intended for gaming.
The A100 also has the distinction of being one of only a few chips to have export controls placed on it because of national defense reasons. Last fall, Nvidia said in an SEC filing that the U.S. government imposed a license requirement barring the export of the A100 and the H100 to China, Hong Kong, and Russia.
“The USG indicated that the new license requirement will address the risk that the covered products may be used in, or diverted to, a ‘military end use’ or ‘military end user’ in China and Russia,” Nvidia said in its filing. Nvidia previously said it adapted some of its chips for the Chinese market to comply with U.S. export restrictions.
The fiercest competition for the A100 may be its successor. The A100 was first introduced in 2020, an eternity ago in chip cycles. The H100, introduced in 2022, is starting to be produced in volume — in fact, Nvidia recorded more revenue from H100 chips in the quarter ending in January than the A100, it said on Wednesday, although the H100 is more expensive per unit.
The H100, Nvidia says, is the first one of its data center GPUs to be optimized for transformers, an increasingly important technique that many of the latest and top AI applications use. Nvidia said on Wednesday that it wants to make AI training over 1 million percent faster. That could mean that, eventually, AI companies wouldn’t need so many Nvidia chips.
Welcome to BizBuzzNews, the premier source of business news and insights for entrepreneurs, executives, and investors. Our mission is to provide you with the latest, most relevant, and insightful news, analysis, and commentary on the world of business.