Richard Yu, Huawei’s chief executive officer, presented 5G equipment on Feb. 25 in Barcelona.

The early waves of mobile communications were largely driven by American and European companies. As the next era of 5G approaches, promising to again transform the way people use the internet, a battle is on to determine whether the U.S. or China will dominate.

Equipment makers and telecom operators in both countries are rushing to test and roll out the next generation of wireless networks, which will be as much as 100 times faster than the current 4G standard. Governments are involved as well—with China making the bigger push.

At the Shenzhen headquarters of Huawei Technologies Co., executives and researchers gathered in July to celebrate one of its technologies being named a critical part of 5G. The man who invented it, Turkish scientist Erdal Arikan, was greeted with thunderous applause. The win meant a stream of future royalties and leverage for the company—and it marked a milestone in China’s quest to dominate the technology.

At a Verizon Communications Inc. lab in Bedminster, N.J., recently, computer screens showed engineers how glare-resistant window coatings can interfere with delivering 5G’s superfast internet into homes. A model of a head known as Mrs. Head tested the audio quality of new wireless devices. Verizon began experimenting with 5G in 11 markets last year.

Nearby, in Murray Hill, N.J., Nokia Corp. engineers are testing a 5G-compatible sleeve that factory workers could wear like an arm brace during their shifts to steer drones or monitor their vital signs. The company began its 5G-related research in 2007.

While the economics of 5G are still being worked out, boosters say the potential payoffs are immense. Companies that own patents stand to make billions of dollars in royalties. Countries with the largest and most reliable networks will have a head start in developing the technologies enabled by faster speeds. The dominant equipment suppliers could give national intelligence agencies and militaries an advantage in spying on or disrupting rival countries’ networks.

“As we face the future, we know deep down that the birth of 5G standards represents a new beginning,” Huawei’s chairman, Eric Xu, told the audience at the company event.

Hans Vestberg, Verizon’s chief executive officer, speaks of the technology in equally dramatic terms. “We are strong believers that 5G [will have] a very transformative effect on many things in our society,” he said. “Consumer, media, entertainment…whole industries.”

By some measures, China is ahead. Since 2013, a government-led committee has worked with China’s mobile carriers and gear-makers on testing and development. The state-led approach, combined with an enormous domestic market, ensures that Chinese companies such as Huawei will sell large quantities of 5G equipment and gain valuable experience in the process.

In the U.S., where the government typically avoids mandating and coordinating efforts by the private sector, much of the experimentation has been led by companies such as AT&TInc., Verizon, Samsung Electronics Co. and Nokia. Last week, tech companies includingIntel Corp. and Cisco Systems Inc. argued in comments filed to the U.S. Trade Representative that proposed tariffs would raise the cost of routers, switches and other goods, slowing development of 5G.

Three of the major carriers plan to roll out 5G service in select cities later this year, though most mobile devices compatible with the new network won’t be ready until early 2019.

The race to 5G has come with tit-for-tat regulatory moves aimed at securing each country’s advantage. In March, the Trump administration blocked Singapore-based Broadcom’s acquisition of U.S. chip giant and 5G leader Qualcomm Inc., citing concerns that Broadcomwould cut the company’s research and development funds and allow Chinese companies to pull ahead in 5G.

In July, China squelched Qualcomm’s planned acquisition of Dutch chip maker NXP Semiconductors NV, a deal that would have helped Qualcomm profit from 5G investments in new markets such as connected cars.

Much of the U.S. unease stems from the rising clout of Huawei, which was labeled a national-security threat, along with ZTE Corp. , by a Congressional panel in 2012 that said those firms’ equipment could be used for spying on Americans. In August, aligning itself with the U.S., Australia said it was banning Huawei and ZTE equipment from its 5G network. Other U.S. allies are studying similar bans.

Huawei and ZTE have consistently denied providing government agencies with backdoor access to their products. Beijing has likewise pushed to replace or sideline U.S. high-tech firms within China’s networks on fears of espionage.

China has made 5G a priority after failing to keep pace with Western countries in developing previous generations of mobile networks. The U.S. dominated 4G, built in the late 2000s, much in the same way Europeans controlled 3G standards. The American lead in 4G has been a boon to companies such as Apple Inc. and Qualcomm, and helped give rise to a host of consumer smartphone applications from the U.S.

Since 2015, China has built about 350,000 cell sites, compared with fewer than 30,000 in the U.S., according to an August study by consulting firm Deloitte. It also noted China has 14.1 sites for every 10,000 people, compared with 4.7 in the U.S. That matters for 5G, because the new networks will require much larger numbers of cell sites than 4G.

The physical manifestation of China’s push is a government-run 5G lab near the Great Wall north of Beijing. The sprawling facility is festooned with base stations and prototype mobile devices, with indoor and outdoor facilities for each of the major Chinese carriers and equipment makers, according to engineers and executives who have visited the site.

Trials are coordinated by a consortium of tech firms, universities and research institutes that operate under China’s Ministry of Industry and Information Technology. The group aims to wrap up tests by the end of the year.

After those trials conclude, state-run carrier China Mobile , the world’s largest mobile operator by subscribers, will follow up with its own tests in 17 cities, according to Chih-Lin I, a former Bell Labs researcher and the company’s chief scientist of mobile technologies. China’s 5G service is expected to be ready for commercial use by 2020.

The faster generation of networks relies on sophisticated technology that allows wireless airwaves to be used more efficiently. Plans call for it to run on high-frequency millimeter waves, which can handle more data but can’t travel as far as lower-frequency waves used by older networks. That means 5G will rely on clusters of antennae as well as decentralized data centers close to consumers and businesses—requiring big investments in infrastructure. The networks are expected to have the speed and responsiveness needed for advances such as driverless cars, which must instantaneously communicate with traffic signals, other cars and their surroundings.

China’s bid to steer the 5G future depends heavily on setting technical standards the rest of the world will have to follow—and pay royalties and licensing fees to use. It has played an aggressive role in the international telecom industry collective that sets global standards.

Experts inside and outside China expect Qualcomm and other Western firms to end up with a majority of the essential patents once the standards are fully determined, but China is making progress.

In 2009, as Huawei’s 5G push began, it recruited Tong Wen, a former senior researcher at now-defunct equipment maker Nortel Networks Corp., to set up a research lab in Ottawa. While flipping through an academic journal, Mr. Tong had stumbled on “polar coding,” a novel method for correcting errors in data transmission invented by Mr. Arikan, the Turkish scientist.

“The Chinese decided this was important,” the expert said. “This was one of the biggest political battles we’ve ever seen.”

The meeting ended with a compromise: Polar codes will be adopted for part of the standard, giving Huawei ownership of a critical patent. The company has spent more than $1 billion on 5G research and development so far.

The U.S. government has stopped short of mandating efforts by the private sector, opening the door to more diffuse outcomes determined by the work of individual companies. In January, a senior National Security Council official floated the idea of rivaling Beijing with a government-led effort to build a nationalized wireless network, but regulators and officials said it was too expensive and unrealistic.

Earlier this month, the Federal Communications Commission announced a plan to speed up the build-out of 5G networks by overriding some local rules and fees governing the deployment of small cellular transmitters, an important component of the infrastructure. The plan is expected to win approval in late September.

The government has funded some academic research that has paved the way for commercial technologies. One agency, the National Science Foundation, is coordinating an effort to build test beds for 5G and future generations of wireless networks.

“The United States is very much behind in this space” relative to Europe, South Korea, Japan and China, said a 2015 internal NSF report on 5G network development.

Thyaga Nandagopal—a former researcher at Bell Labs who is a director at the foundation—is leading the test bed project, in which companies, academics and government agencies will be able to test 5G and other wireless network applications in tandem. Nearly 30 U.S., European and Asian companies have committed $50 million of capital and equipment over the next seven years, while the U.S. government has pledged to invest another $50 million. In New York, an NSF-funded site run by academic institutions including Columbia University aims to launch a small pilot phase by the beginning of January.

Mr. Nandagopal said that China’s coordinated investments have put it in a “pretty good pole position” but that the NSF’s efforts are focused on wireless developments after 2020, rather than the early years of 5G deployment.

“We can invest our money strategically and still get better results than anyone else,” he said.

Some American telecom companies are staking claims to rooftops and light poles where they can position small cells that enable the faster networks, and pressing equipment and device makers to create 5G-compatible products.

For all the investment, industry experts note the standards for 5G aren’t fully written and wireless carriers are still figuring out how they can best profit from the service.

At a 5G forum in Santa Clara, Calif., in July, Henning Schulzrinne, a former chief technology officer at the FCC, said operators would also have to find a way to drastically reduce the cost of data to make applications such as augmented or virtual reality affordable enough to sell to consumers over 5G. Some of those applications could work using 4G or Wi-Fi instead.

“Who’s going to stream AR or VR if it’s going to cost them $10 per minute?” he said.

John Donovan, chief executive of AT&T’s communications business, said the company’s researchers have been among the most prolific writers of 5G standards, but it is being cautious as it puts the technology in the field.

“To deploy technology in advance of need, before the use cases are there—you’re wasting money,” he said.

Executives at Huawei have also sought to temper 5G expectations. Before an audience of analysts at an annual meeting at the Shenzhen headquarters in April, Mr. Xu, Huawei’s chairman, said that “the entire industry and also governments around the world have regarded 5G too high, to the extent that it’s going to be the digital infrastructure for everything.”

Huawei and China Mobile will push ahead with 5G on a large scale regardless, according to executives from both companies.

“5G is such an important strategic project for China—kitchen sink, all the resources,” said Edison Lee, a telecom analyst at investment bank Jefferies in Hong Kong. “Because if they get their foot in the door for 5G, they get their foot in the door of 6G, 7G, 8G.”