It’s hard to think of a more dramatic way to make an entrance than falling from the sky. While it certainly happens often enough on the silver screen, whether or not it can be done in real life is a tantalizing challenge for our entertainment robotics team at Disney Research.

Falling is tricky for two reasons. The first and most obvious is what Douglas Adams referred to as “the sudden stop at the end.” Every second of free fall means another 9.8 m/s of velocity, and that can quickly add up to an extremely difficult energy dissipation problem. The other tricky thing about falling, especially for terrestrial animals like us, is that our normal methods for controlling our orientation disappear. We are used to relying on contact forces between our body and the environment to control which way we’re pointing. In the air, there’s nothing to push on except the air itself!

Finding a solution to these problems is a big, open-ended challenge. In the clip below, you can see one approach we’ve taken to start chipping away at it.

The video shows a small, stick-like robot with an array of four ducted fans attached to its top. The robot has a piston-like foot that absorbs the impact of a small fall, and then the ducted fans keep the robot standing by counteracting any tilting motion using aerodynamic thrust.

Raphael Pilon [left] and Marcela de los Rios evaluate the performance of the monopod balancing robot.Disney Research

The standing portion demonstrates that pushing on the air isn’t only useful during freefall. Conventional walking and hopping robots depend on ground contact forces to maintain the required orientation. These forces can ramp up quickly because of the stiffness of the system, necessitating high bandwidth control strategies. Aerodynamic forces are relatively soft, but even so, they were sufficient to keep our robots standing. And since these forces can also be applied during the flight phase of running or hopping, this approach might lead to robots that run before they walk. The thing that defines a running gait is the existence of a “flight phase” - a time when none of the feet are in contact with the ground. A running robot with aerodynamic control authority could potentially use a gait with a long flight phase. This would shift the burden of the control effort to mid-flight, simplifying the leg design and possibly making rapid bipedal motion more tractable than a moderate pace.

Richard Landon uses a test rig to evaluate the thrust profile of a ducted fan.Disney Research

In the next video, a slightly larger robot tackles a much more dramatic fall, from 65’ in the air. This simple machine has two piston-like feet and a similar array of ducted fans on top. The fans not only stabilize the robot upon landing, they also help keep it oriented properly as it falls. Inside each foot is a plug of single-use compressible foam. Crushing the foam on impact provides a nice, constant force profile, which maximizes the amount of energy dissipated per inch of contraction.

In the case of this little robot, the mechanical energy dissipation in the pistons is less than the total energy needed to be dissipated from the fall, so the rest of the mechanism takes a pretty hard hit. The size of the robot is an advantage in this case, because scaling laws mean that the strength-to-weight ratio is in its favor.

The strength of a component is a function of its cross-sectional area, while the weight of a component is a function of its volume. Area is proportional to length squared, while volume is proportional to length cubed. This means that as an object gets smaller, its weight becomes relatively small. This is why a toddler can be half the height of an adult but only a fraction of that adult’s weight, and why ants and spiders can run around on long, spindly legs. Our tiny robots take advantage of this, but we can’t stop there if we want to represent some of our bigger characters.

Louis Lambie and Michael Lynch assemble an early ducted fan test platform. The platform was mounted on guidewires and was used for lifting capacity tests.Disney Research

In most aerial robotics applications, control is provided by a system that is capable of supporting the entire weight of the robot. In our case, being able to hover isn’t a necessity. The clip below shows an investigation into how much thrust is needed to control the orientation of a fairly large, heavy robot. The robot is supported on a gimbal, allowing it to spin freely. At the extremities are mounted arrays of ducted fans. The fans don’t have enough force to keep the frame in the air, but they do have a lot of control authority over the orientation.

Complicated robots are less likely to survive unscathed when subjected to the extremely high accelerations of a direct ground impact, as you can see in this early test that didn’t quite go according to plan.

In this last video, we use a combination of the previous techniques and add one more capability – a dramatic mid-air stop. Ducted fans are part of this solution, but the high-speed deceleration is principally accomplished by a large water rocket. Then the mechanical legs only have to handle the last ten feet of dropping acceleration.

Whether it’s using water or rocket fuel, the principle underlying a rocket is the same – mass is ejected from the rocket at high speed, producing a reaction force in the opposite direction via Newton’s third law. The higher the flow rate and the denser the fluid, the more force is produced. To get a high flow rate and a quick response time, we needed a wide nozzle that went from closed to open cleanly in a matter of milliseconds. We designed a system using a piece of copper foil and a custom punch mechanism that accomplished just that.

Grant Imahara pressurizes a test tank to evaluate an early valve prototype [left]. The water rocket in action - note the laminar, two-inch-wide flow as it passes through the specially designed nozzleDisney Research

Once the water rocket has brought the robot to a mid-air stop, the ducted fans are able to hold it in a stable hover about ten feet above the deck. When they cut out, the robot falls again and the legs absorb the impact. In the video, the robot has a couple of loose tethers attached as a testing precaution, but they don’t provide any support, power, or guidance.

“It might not be so obvious as to what this can be directly used for today, but these rough proof-of-concept experiments show that we might be able to work within real-world physics to do the high falls our characters do on the big screen, and someday actually stick the landing,” explains Tony Dohi, the project lead.

There are still a large number of problems for future projects to address. Most characters have legs that bend on hinges rather than compress like pistons, and don’t wear a belt made of ducted fans. Beyond issues of packaging and form, making sure that the robot lands exactly where it intends to land has interesting implications for perception and control. Regardless, we think we can confirm that this kind of entrance has–if you’ll excuse the pun–quite the impact.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

ICRA 2024: 13–17 May 2024, YOKOHAMA, JAPAN

RoboCup 2024: 17–22 July 2024, EINDHOVEN, NETHERLANDS

ICSR 2024: 23–26 October 2024, ODENSE, DENMARK

Cybathlon 2024: 25–27 October 2024, ZURICH

Enjoy today’s videos!

Festo has robot bees!

It’s a very clever design, but the size makes me terrified of whatever the bees are that Festo seems to be familiar with.

[ Festo ]

Boing, boing, boing!

[ USC ]

Why the heck would you take the trouble to program a robot to make sweet potato chips and then not scarf them down yourself?

[ Dino Robotics ]

Mobile robots can transport payloads far greater than their mass through vehicle traction. However, off-road terrain features substantial variation in height, grade, and friction, which can cause traction to degrade or fail catastrophically. This paper presents a system that utilizes a vehicle-mounted, multipurpose manipulator to physically adapt the robot with unique anchors suitable for a particular terrain for autonomous payload transport.

[ DART Lab ]

Turns out that working on a collaborative task with a robot can make humans less efficient, because we tend to overestimate the robot’s capabilities.

[ CHI 2024 ]

Wing posts a video with the title “What Do Wing’s Drones Sound Like” but only includes a brief snippet—though nothing without background room noise—revealing to curious viewers and listeners exactly what Wing’s drones sound like.

Because, look, a couple seconds of muted audio underneath a voiceover is in fact not really answering the question.

[ Wing ]

This first instance of ROB 450 in Winter 2024 challenged students to synthesize the knowledge acquired through their Robotics undergraduate courses at the University of Michigan to use a systematic and iterative design and analysis process and apply it to solving a real, open-ended Robotics problem.

[ Michigan Robotics ]

This Microsoft Future Leaders in Robotics and AI Seminar is from Catie Cuan at Stanford, on “Choreorobotics: Teaching Robots How to Dance With Humans.”

As robots transition from industrial and research settings into everyday environments, robots must be able to (1) learn from humans while benefiting from the full range of the humans’ knowledge and (2) learn to interact with humans in safe, intuitive, and social ways. I will present a series of compelling robot behaviors, where human perception and interaction are foregrounded in a variety of tasks.

[ UMD ]

For years, Shadow Robot Company’s Shadow Hand has arguably been the gold standard for robotic manipulation. Beautiful and expensive, it is able to mimic the form factor and functionality of human hands, which has made it ideal for complex tasks. I’ve personally experienced how amazing it is to use Shadow Hands in a teleoperation context, and it’s hard to imagine anything better.

The problem with the original Shadow hand was (and still is) fragility. In a research environment, this has been fine, except that research is changing: Roboticists no longer carefully program manipulation tasks by, uh, hand. Now it’s all about machine learning, in which you need robotic hands to massively fail over and over again until they build up enough data to understand how to succeed.

“We’ve aimed for robustness and performance over anthropomorphism and human size and shape.” —Rich Walker, Shadow Robot Company

Doing this with a Shadow Hand was just not realistic, which Google DeepMind understood five years ago when it asked Shadow Robot to build it a new hand with hardware that could handle the kind of training environments that now typify manipulation research. So Shadow Robot spent the last half-decade-ish working on a new, three-fingered Shadow Hand, which the company unveiled today. The company is calling it, appropriately enough, “the new Shadow Hand.”

As you can see, this thing is an absolute beast. Shadow Robot says that the new hand is “robust against a significant amount of misuse, including aggressive force demands, abrasion and impacts.” Part of the point, though, is that what robot-hand designers might call “misuse,” robot-manipulation researchers might very well call “progress,” and the hand is designed to stand up to manipulation research that pushes the envelope of what robotic hardware and software are physically capable of.

Shadow Robot understands that despite its best engineering efforts, this new hand will still occasionally break (because it’s a robot and that’s what robots do), so the company designed it to be modular and easy to repair. Each finger is its own self-contained unit that can be easily swapped out, with five Maxon motors in the base of the finger driving the four finger joints through cables in a design that eliminates backlash. The cables themselves will need replacement from time to time, but it’s much easier to do this on the new Shadow Hand than it was on the original. Shadow Robot says that you can swap out an entire New Hand’s worth of cables in the same time it would take you to replace a single cable on the old hand.

Shadow Robot

The new Shadow Hand itself is somewhat larger than a typical human hand, and heavier too: Each modular finger unit weighs 1.2 kilograms, and the entire three-fingered hand is just over 4 kg. The fingers have humanlike kinematics, and each joint can move up to 180 degrees per second with the capability of exerting at least 8 newtons of force at each fingertip. Both force control and position control are available, and the entire hand runs Robot Operating System, the Open Source Robotics Foundation’s collection of open-source software libraries and tools.

One of the coolest new features of this hand is the tactile sensing. Shadow Robot has decided to take the optical route with fingertip sensors, GelSight-style. Each fingertip is covered in soft, squishy gel with thousands of embedded particles. Cameras in the fingers behind the gel track each of those particles, and when the fingertip touches something, the particles move. Based on that movement, the fingertips can very accurately detect the magnitude and direction of even very small forces. And there are even more sensors on the insides of the fingers too, with embedded Hall effect sensors to help provide feedback during grasping and manipulation tasks.

Shadow Robot

The most striking difference here is how completely different of a robotic-manipulation philosophy this new hand represents for Shadow Robot. “We’ve aimed for robustness and performance over anthropomorphism and human size and shape,” says Rich Walker, director of Shadow Robot Company. “There’s a very definite design choice there to get something that really behaves much more like an optimized manipulator rather than a humanlike hand.”

Walker explains that Shadow Robot sees two different approaches to manipulation within the robotics community right now: There’s imitation learning, where a human does a task and then a robot tries to do the task the same way, and then there’s reinforcement learning, where a robot tries to figure out how do the task by itself. “Obviously, this hand was built from the ground up to make reinforcement learning easy.”

The hand was also built from the ground up to be rugged and repairable, which had a significant effect on the form factor. To make the fingers modular, they have to be chunky, and trying to cram five of them onto one hand was just not practical. But because of this modularity, Shadow Robot could make you a five-fingered hand if you really wanted one. Or a two-fingered hand. Or (and this is the company’s suggestion, not mine) “a giant spider.” Really, though, it’s probably not useful to get stuck on the form factor. Instead, focus more on what the hand can do. In fact, Shadow Robot tells me that the best way to think about the hand in the context of agility is as having three thumbs, not three fingers, but Walker says that “if we describe it as that, people get confused.”

There’s still definitely a place for the original anthropomorphic Shadow Hand, and Shadow Robot has no plans to discontinue it. “It’s clear that for some people anthropomorphism is a deal breaker, they have to have it,” Walker says. “But for a lot of people, the idea that they could have something which is really robust and dexterous and can gather lots of data, that’s exciting enough to be worth saying okay, what can we do with this? We’re very interested to find out what happens.”

The Shadow New Hand is available now, starting at about US $74,000 depending on configuration.

A changing of the guard in space stations is on the horizon as private companies work toward providing new opportunities for science, commerce, and tourism in outer space.

Blue Origin is one of a number of private-sector actors aiming to harbor commercial activities in low Earth orbit (LEO) as the creaking and leaking International Space Station (ISS) approaches its drawdown. Partners in Blue Origin’s Orbital Reef program, including firms Redwire, Sierra Space, and Boeing, are each reporting progress in their respective components of the program. The collaboration itself may not be on such strong ground. Such endeavors may also end up slowed and controlled by regulation so far absent from many new, commercial areas of space.

Orbital Reef recently aced testing milestones for its critical life support system, with assistance from NASA. These included hitting targets for trace contaminant control, water contaminant oxidation, urine water recovery, and water tank tests—all of which are required to operate effectively and efficiently to enable finite resources to keep delicate human beings alive in orbit for long timeframes.

Blue Origin, founded by Jeff Bezos, is characteristically tight-lipped on its progress and challenges and declined to provide further comment on progress beyond NASA’s life-support press statement.

The initiative is backed by NASA’s Commercial LEO Destinations (CLD) program, through which the agency is providing funding to encourage the private sector to build space habitats. NASA may also be the main client starting out, although the wider goal is to foster a sustainable commercial presence in LEO.

The Space-Based Road Ahead

The challenge Orbital Reef faces is considerable: reimagining successful earthbound technologies—such as regenerative life-support systems, expandable habitats and 3D printing—but now in orbit, on a commercially viable platform. The technologies must also adhere to unforgiving constraints of getting mass and volume to space, and operating on a significantly reduced budget compared to earlier national space station programs.

Add to that autonomy and redundancy that so many mission-critical functions will demand, as well as high-bandwidth communications required to return data and allow streaming and connectivity for visitors.

In one recent step forward for Orbital Reef, Sierra Space, headquartered in Louisville, Colo., performed an Ultimate Burst Pressure (UBP) test on its architecture in January. This involved inflating, to the point of failure, the woven fabric pressure shell—including Vectran, a fabric that becomes rigid and stronger than steel when pressurized on orbit—for its Large Integrated Flexible Environment (LIFE) habitat. Sierra’s test reached 530,000 pascals (77 pounds per square inch) before it burst—marking a successful failure that far surpassed NASA’s recommended safety level of 419,200 Pa (60.8 psi).

Notably, the test article was 300 cubic meters in volume, or one-third the volume of ISS—a megaproject constructed by some 15 countries over more than 30 launches. LIFE will contain 10 crew cabins along with living, galley, and gym areas. This is expected to form part of the modular Orbital Reef complex. The company stated last year it aimed to launch a pathfinder version of LIFE around the end of 2026.

Inflating and Expanding Expectations

Whereas the size of ISS modules and those of China’s new, three-module Tiangong space station, constructed in 2021–22, was dependent on the size of the payload bay or fairing of the shuttle or rocket doing the launching, using expandable quarters allows Orbital Reef to offer habitable areas multiples (in this case five times) greater than the volume of the 5-meter rocket fairing to be used to transport the system to orbit.

Other modules will include Node, with an airlock and docking facilities, also developed by Sierra Space, as well as a spherical Core module developed by Blue Origin. Finally, Boeing is developing a research module, which will include a science cupola, akin to that on the ISS, external payload facilities, and a series of laboratories.

Orbital Reef will be relying on some technologies developed for and spun off from the ISS project, which was completed in 2011 at a cost of US $100 billion. The new station will be operating on fractions of such budgets, with Blue Origin awarded $130 million of a total $415.6 million given to three companies in 2021.

“NASA is using a two-phase strategy to, first, support the development of commercial destinations and, secondly, enable the agency to purchase services as one of many customers” says NASA spokesperson Anna Schneider, at NASA’s Johnson Space Center.

For instance, Northrop Grumman is working on its Persistent Platform to provide autonomous and robotic capabilities for commercial science and manufacturing capabilities in LEO.

Such initiatives could face politically constructed hurdles, however. Last year, some industry advocates opposed a White House proposal that would see new commercial space activities such as space stations regulated.

Meanwhile, the European Space Agency (ESA) signed a memorandum of understanding in late 2023 with Airbus and Voyager Space, headquartered in Denver, which would give ESA access to a planned Starlab space station after the ISS is transitioned out. That two-module orbital outpost will also be inflatable and is now expected to be launched in 2028.

China also is exploring opening its Tiangong station to commercial activities, including its own version of NASA’s commercial cargo and extending the station with new modules—and new competition for the world’s emerging space station sector.

This article is part of our exclusive career advice series in partnership with the IEEE Technology and Engineering Management Society.

As you begin your professional career freshly armed with an engineering degree, your initial roles and responsibilities are likely to revolve around the knowledge and competencies you learned at school. If you do well in your job, you’re apt to be promoted, gaining more responsibilities such as managing projects, interacting with other departments, making presentations to management, and meeting with customers. You probably also will gain a general understanding of how your company and the business world work.

At some point in your career, you’re likely to be asked an important question: Are you interested in a management role?

There is no right or wrong answer. Engineers have fulfilling, rewarding careers as individual contributors and as managers—and companies need both. You should decide your path based on your interests and ambitions as well as your strengths and shortcomings.

However, the specific considerations involved aren’t always obvious. To help you, this article covers some of the differences between the two career paths, as well as factors that might influence you.

The remarks are based on our personal experiences in corporate careers spanning decades in the managerial track and the technical track. Tariq worked at Honeywell; Gus at 3M. We have included advice from IEEE Technology and Engineering Management Society colleagues.

Opting for either track isn’t a career-long commitment. Many engineers who go into management return to the technical track, in some cases of their own volition. And management opportunities can be adopted late in one’s career, again based on individual preferences or organizational needs.

In either case, there tends to be a cost to switching tracks. While the decision of which track to take certainly isn’t irrevocable, it behooves engineers to understand the pros and cons involved.

Differences between the two tracks

Broadly, the managerial track is similar across all companies. It starts with supervising small groups, extends through middle-management layers, progresses up to leadership positions and, ultimately, the executive suite. Management backgrounds can vary, however. For example, although initial management levels in a technology organization generally require an engineering or science degree, some top leaders in a company might be more familiar with sales, marketing, or finance.

It’s a different story for climbing the technical ladder. Beyond the first engineering-level positions, there is no standard model. In some cases individual contributors hit the career ceiling below the management levels. In others, formal roles exist that are equivalent to junior management positions in terms of pay scale and other aspects.

“Engineers have fulfilling, rewarding careers as individual contributors and as managers—and companies need both.”

Some organizations have a well-defined promotional system with multiple salary bands for technical staff, parallel to those for management positions. Senior technologists often have a title such as Fellow, staff scientist, or architect, with top-of-the-ladder positions including corporate Fellow, chief engineer/scientist, and enterprise architect.

Organizational structures vary considerably among small companies—including startups, medium companies, and large corporations. Small businesses often don’t have formal or extensive technical tracks, but their lack of structure can make it easier to advance in responsibilities and qualifications while staying deeply technical.

In more established companies, structures and processes tend to be well defined and set by policy.

For those interested in the technical track, the robustness of a company’s technical ladder can be a factor in joining the company. Conversely, if you’re interested in the technical ladder and you’re working for a company that does not offer one, that might be a reason to look for opportunities elsewhere.

Understanding the career paths a company offers is especially important for technologists.

The requirements for success

First and foremost, the track you lean toward should align with aspirations for your career—and your personal life.

As you advance in the management path, you can drive business and organizational success through decisions you make and influence. You also will be expected to shape and nurture employees in your organization by providing feedback and guidance. You likely will have more control over resources—people as well as funding—and more opportunity for defining and executing strategy.

The technical path has much going for it as well, especially if you are passionate about solving technical challenges and increasing your expertise in your area of specialization. You won’t be supervising large numbers of employees, but you will manage significant projects and programs that give you chances to propose and define such initiatives. You also likely will have more control of your time and not have to deal with the stress involved with being responsible for the performance of the people and groups reporting to you.

The requirements for success in the two tracks offer contrasts as well. Technical expertise is an entry requirement for the technical track. It’s not just technical depth, however. As you advance, technical breadth is likely to become increasingly important and will need to be supplemented by an understanding of the business, including markets, customers, economics, and government regulations.

Pure technical expertise will never be the sole performance criterion. Soft skills such as verbal and written communication, getting along with people, time management, and teamwork are crucial for managers and leaders.

On the financial side, salaries and growth prospects generally will be higher on the managerial track. Executive tiers can include substantial bonuses and stock options. Salary growth is typically slower for senior technologists.

Managerial and technical paths are not always mutually exclusive. It is, in fact, not uncommon for staff members who are on the technical ladder to supervise small teams. And some senior managers are able to maintain their technical expertise and earn recognition for it.

We recommend you take time to consider which of the two tracks is more attractive—before you get asked to choose. If you’re early in your career, you don’t need to make this important decision now. You can keep your options open and discuss them with your peers, senior colleagues, and management. And you can contemplate and clarify what your objectives and preferences are. When the question does come up, you’ll be better prepared to answer it.

Connected vehicles offer a range of benefits, such as real-time data sharing, app-to-car connectivity, advanced driver assistance systems (ADAS), and critical safety features like location tracking, remote parking, and in-vehicle infotainment systems (IVIs). These advancements aim to enhance the overall driving and riding experience. However, it is crucial to acknowledge that equipping vehicles with smart features also exposes them to potential cyberattacks. These attacks can result in customer data leakage or even compromise critical safety functionalities.

It’s expected to discover vulnerabilities after the product is released, which could have been easily prevented. For instance, as reported by Bloomberg, a recent increase in car thefts was attributed to the absence of anti-theft computer chips in vehicle critical systems. Therefore, it is imperative to proactively consider and address potential attack vectors right from the initial stages of development. This cybersecurity vulnerability applies to many other industrial applications, such as industrial IoT, SmartCity, and digital healthcare, where every device or system is connected, and every connection is a vulnerability.

Design for security is becoming mainstream and should be part of today’s standard design methodologies.

What you will learn:

1. Why a model-based and system-oriented solution is needed for automotive cybersecurity

2. How to quickly identify threat scenarios

3. Why a pre-silicon security verification flow is essential for secure ICs

4. Using AI to mitigate side-channel vulnerabilities

Who should attend this presentation:

This webinar is valuable to anyone who works with product design, connectivity and security.

Register now for this free webinar!

The accelerating buildout of solar farms on Earth is already hitting speed bumps, including public pushback against the large tracts of land required and a ballooning backlog of requests for new transmission lines and grid connections. Energy experts have been warning that electricity is likely to get more expensive and less reliable unless renewable power that waxes and wanes under inconstant sunlight and wind is backed up by generators that can run whenever needed. To space enthusiasts, that raises an obvious question: Why not stick solar power plants where the sun always shines?

Space-based solar power is an idea so beautiful, so tantalizing that some argue it is a wish worth fulfilling. A constellation of gigantic satellites in geosynchronous orbit (GEO) nearly 36,000 kilometers above the equator could collect sunlight unfiltered by atmosphere and uninterrupted by night (except for up to 70 minutes a day around the spring and fall equinoxes). Each megasat could then convert gigawatts of power into a microwave beam aimed precisely at a big field of receiving antennas on Earth. These rectennas would then convert the signal to usable DC electricity.

The thousands of rocket launches needed to loft and maintain these space power stations would dump lots of soot, carbon dioxide, and other pollutants into the stratosphere, with uncertain climate impacts. But that might be mitigated, in theory, if space solar displaced fossil fuels and helped the world transition to clean electricity.

The glamorous vision has inspired numerous futuristic proposals. Japan’s space agency has presented a road map to deployment. Space authorities in China aim to put a small test satellite in low Earth orbit (LEO) later this decade. Ideas to put megawatt-scale systems in GEO sometime in the 2030s have been floated but not yet funded.

The U.S. Naval Research Laboratory has already beamed more than a kilowatt of power between two ground antennas about a kilometer apart. It also launched in 2023 a satellite that used a laser to transmit about 1.5 watts, although the beam traveled less than 2 meters and the system had just 11 percent efficiency. A team at Caltech earlier this year wrapped up a mission that used a small satellite in LEO to test thin-film solar cells, flexible microwave-power circuitry, and a small collapsible deployment mechanism. The energy sent Earthward by the craft was too meager to power a lightbulb, but it was progress nonetheless.

The European Space Agency (ESA) debuted in 2022 its space-based solar-power program, called Solaris, with an inspiring (but entirely fantastical) video animation. The program’s director, Sanjay Vijendran, told IEEE Spectrum that the goal of the effort is not to develop a power station for space. Instead, the program aims to spend three years and €60 million (US $65 million) to figure out whether solar cells, DC-to-RF converters, assembly robots, beam-steering antennas, and other must-have technologies will improve drastically enough over the next 10 to 20 years to make orbital solar power feasible and competitive. Low-cost, low-mass, and space-hardy versions of these technologies would be required, but engineers trying to draw up detailed plans for such satellites today find no parts that meet the tough requirements.

Not so fast: The real-world efficiency of commercial, space-qualified solar cells has progressed much more slowly than records set in highly controlled research experiments, which often use exotic materials or complex designs that cannot currently be mass-produced. Points plotted here show the highest efficiency reported in five-year intervals.HENRI BARDE; DATA FROM NATIONAL RENEWABLE ENERGY LABORATORY (RESEARCH CELLS) AND FROM MANUFACTURER DATA SHEETS AND PRESENTATIONS (COMMERCIAL CELLS)

With the flurry of renewed attention, you might wonder: Has extraterrestrial solar power finally found its moment? As the recently retired head of space power systems at ESA—with more than 30 years of experience working on power generation, energy storage, and electrical systems design for dozens of missions, including evaluation of a power-beaming experiment proposed for the International Space Station—I think the answer is almost certainly no.

Despite mounting buzz around the concept, I and many of my former colleagues at ESA are deeply skeptical that these large and complex power systems could be deployed quickly enough and widely enough to make a meaningful contribution to the global energy transition. Among the many challenges on the long and formidable list of technical and societal obstacles: antennas so big that we cannot even simulate their behavior.

Here I offer a road map of the potential chasms and dead ends that could doom a premature space solar project to failure. Such a misadventure would undermine the credibility of the responsible space agency and waste capital that could be better spent improving less risky ways to shore up renewable energy, such as batteries, hydrogen, and grid improvements. Champions of space solar power could look at this road map as a wish list that must be fulfilled before orbital solar power can become truly appealing to electrical utilities.

Space Solar Power at Peak Hype—Again

For decades, enthusiasm for the possibility of drawing limitless, mostly clean power from the one fusion reactor we know works reliably—the sun—has run hot and cold. A 1974 study that NASA commissioned from the consultancy Arthur D. Little bullishly recommended a 20-year federal R&D program, expected to lead to a commercial station launching in the mid-1990s. After five years of work, the agency delivered a reference architecture for up to 60 orbiting power stations, each delivering 5 to 10 gigawatts of baseload power to major cities. But officials gave up on the idea when they realized that it would cost over $1 trillion (adjusted for inflation) and require hundreds of astronauts working in space for decades, all before the first kilowatt could be sold.

NASA did not seriously reconsider space solar until 1995, when it ordered a “fresh look” at the possibility. That two-year study generated enough interest that the U.S. Congress funded a small R&D program, which published plans to put up a megawatt-scale orbiter in the early 2010s and a full-size power plant in the early 2020s. Funding was cut off a few years later, with no satellites developed.

Because of the physics of power transmission from geosynchronous orbit, space power satellites must be enormous—hundreds of times larger than the International Space Station and even dwarfing the tallest skyscrapers—to generate electricity at a competitive price. The challenges for their engineering and assembly are equally gargantuan. Chris Philpot

Then, a decade ago, private-sector startups generated another flurry of media attention. One, Solaren, even signed a power-purchase agreement to deliver 200 megawatts to utility customers in California by 2016 and made bold predictions that space solar plants would enter mass production in the 2020s. But the contract and promises went unfulfilled.

The repeated hype cycles have ended the same way each time, with investors and governments balking at the huge investments that must be risked to build a system that cannot be guaranteed to work. Indeed, in what could presage the end of the current hype cycle, Solaris managers have had trouble drumming up interest among ESA’s 22 member states. So far only the United Kingdom has participated, and just 5 percent of the funds available have been committed to actual research work.

Even space-solar advocates have recognized that success clearly hinges on something that cannot be engineered: sustained political will to invest, and keep investing, in a multidecade R&D program that ultimately could yield machines that can’t put electricity on the grid. In that respect, beamed power from space is like nuclear fusion, except at least 25 years behind.

In the 1990s, the fusion community succeeded in tapping into national defense budgets and cobbled together the 35-nation, $25 billion megaproject ITER, which launched in 2006. The effort set records for delays and cost overruns, and yet a prototype is still years from completion. Nevertheless, dozens of startups are now testing new fusion-reactor concepts. Massive investments in space solar would likely proceed in the same way. Of course, if fusion succeeds, it would eclipse the rationale for solar-energy satellites.

Space Industry Experts Run the Numbers

The U.S. and European space agencies have recently released detailed technical analyses of several space-based solar-power proposals. [See diagrams.] These reports make for sobering reading.

SPS-ALPHA Mark-III

Proposed by: John C. Mankins, former NASA physicist

Features: Thin-film reflectors (conical array) track the sun and concentrate sunlight onto an Earth-facing energy-conversion array that has photovoltaic (PV) panels on one side, microwave antennas on the other, and power distribution and control electronics in the middle. Peripheral modules adjust the station’s orbit and orientation.

MR-SPS

Proposed by: China Academy of Space Technology

Features: Fifty PV solar arrays, each 200 meters wide and 600 meters long, track the sun and send power through rotating high-power joints and perpendicular trusses to a central microwave-conversion and transmission array that points 128,000 antenna modules at the receiving station on Earth.

CASSIOPeiA

Proposed by: Ian Cash, chief architect, Space Solar Group Holdings

Features: Circular thin-film reflectors track the sun and bounce light onto a helical array that includes myriad small PV cells covered by Fresnel-lens concentrators, power-conversion electronics, and microwave dipole antennas. The omnidirectional antennas must operate in sync to steer the beam as the station rotates relative to the Earth.

SPS (Solar power satellite)

Proposed by: Thales Alenia Space

Features: Nearly 8,000 flexible solar arrays, each 10 meters wide and 80 meters long, are unfurled from roll-out modules and linked together to form two wings. The solar array remains pointed at the sun, so the central transmitter must rotate and also operate with great precision as a phased-array antenna to continually steer the beam onto the ground station.

Electricity made this way, NASA reckoned in its 2024 report, would initially cost 12 to 80 times as much as power generated on the ground, and the first power station would require at least $275 billion in capital investment. Ten of the 13 crucial subsystems required to build such a satellite—including gigawatt-scale microwave beam transmission and robotic construction of kilometers-long, high-stiffness structures in space—rank as “high” or “very high” technical difficulty, according to a 2022 report to ESA by Frazer-Nash, a U.K. consultancy. Plus, there is no known way to safely dispose of such enormous structures, which would share an increasingly crowded GEO with crucial defense, navigation, and communications satellites, notes a 2023 ESA study by the French-Italian satellite maker Thales Alenia Space.

An alternative to microwave transmission would be to beam the energy down to Earth as reflected sunlight. Engineers at Arthur D. Little described the concept in a 2023 ESA study in which they proposed encircling the Earth with about 4,000 aimable mirrors in LEO. As each satellite zips overhead, it would shine an 8-km-wide spotlight onto participating solar farms, allowing the farms to operate a few extra hours each day (if skies are clear). In addition to the problems of clouds and light pollution, the report noted the thorny issue of orbital debris, estimating that each reflector would be penetrated about 75 billion times during its 10-year operating life.

My own assessment, presented at the 2023 European Space Power Conference and published by IEEE, pointed out dubious assumptions and inconsistencies in four space-solar designs that have received serious attention from government agencies. Indeed, the concepts detailed so far all seem to stand on shaky technical ground.

Massive Transmitters and Receiving Stations

The high costs and hard engineering problems that prevent us from building orbital solar-power systems today arise mainly from the enormity of these satellites and their distance from Earth, both of which are unavoidable consequences of the physics of this kind of energy transmission. Only in GEO can a satellite stay (almost) continuously connected to a single receiving station on the ground. The systems must beam down their energy at a frequency that passes relatively unimpeded through all kinds of weather and doesn’t interfere with critical radio systems on Earth. Most designs call for 2.45 or 5.8 gigahertz, within the range used for Wi-Fi. Diffraction will cause the beam to spread as it travels, by an amount that depends on the frequency.

Thales Alenia Space estimated that a transmitter in GEO must be at least 750 meters in diameter to train the bright center of a 5.8-GHz microwave beam onto a ground station of reasonable area over that tremendous distance—65 times the altitude of LEO satellites like Starlink. Even using a 750-meter transmitter, a receiver station in France or the northern United States would fill an elliptical field covering more than 34 square kilometers. That’s more than two-thirds the size of Bordeaux, France, where I live.

“Success hinges on something that cannot be engineered: sustained political will to keep investing in a multidecade R&D program that ultimately could yield machines that can’t put electricity on the grid.”

Huge components come with huge masses, which lead to exorbitant launch costs. Thales Alenia Space estimated that the transmitter alone would weigh at least 250 tonnes and cost well over a billion dollars to build, launch, and ferry to GEO. That estimate, based on ideas from the Caltech group that have yet to be tested in space, seems wildly optimistic; previous detailed transmitter designs are about 30 times heavier.

Because the transmitter has to be big and expensive, any orbiting solar project will maximize the power it sends through the beam, within acceptable safety limits. That’s why the systems evaluated by NASA, ESA, China, and Japan are all scaled to deliver 1–2 GW, the maximum output that utilities and grid operators now say they are willing to handle. It would take two or three of these giant satellites to replace one large retiring coal or nuclear power station.

Energy is lost at each step in the conversion from sunlight to DC electricity, then to microwaves, then back to DC electricity and finally to a grid-compatible AC current. It will be hard to improve much on the 11 percent end-to-end efficiency seen in recent field trials. So the solar arrays and electrical gear must be big enough to collect, convert, and distribute around 9 GW of power in space just to deliver 1 GW to the grid. No electronic switches, relays, and transformers have been designed or demonstrated for spacecraft that can handle voltages and currents anywhere near the required magnitude.

Some space solar designs, such as SPS-ALPHA and CASSIOPeiA, would suspend huge reflectors on kilometers-long booms to concentrate sunlight onto high-efficiency solar cells on the back side of the transmitter or intermingled with antennas. Other concepts, such as China’s MR-SPS and the design proposed by Thales Alenia Space, would send the currents through heavy, motorized rotating joints that allow the large solar arrays to face the sun while the transmitter pivots to stay fixed on the receiving station on Earth.

All space solar-power concepts that send energy to Earth via a microwave beam would need a large receiving station on the ground. An elliptical rectenna field 6 to 10 kilometers wide would be covered with antennas and electronics that rectify the microwaves into DC power. Additional inverters would then convert the electricity to grid-compatible AC current.Chris Philpot

The net result, regardless of approach, is an orbiting power station that spans several kilometers, totals many thousands of tonnes, sends gigawatts of continuous power through onboard electronics, and comprises up to a million modules that must be assembled in space—by robots. That is a gigantic leap from the largest satellite and solar array ever constructed in orbit: the 420-tonne, 109-meter International Space Station (ISS), whose 164 solar panels produce less than 100 kilowatts to power its 43 modules.

The ISS has been built and maintained by astronauts, drawing on 30 years of prior experience with the Salyut, Skylab, and Mir space stations. But there is no comparable incremental path to a robot-assembled power satellite in GEO. Successfully beaming down a few megawatts from LEO would be an impressive achievement, but it wouldn’t prove that a full-scale system is feasible, nor would the intermittent power be particularly interesting to commercial utilities.

T Minus...Decades?

NASA’s 2024 report used sensitivity analysis to look for advances, however implausible, that would enable orbital solar power to be commercially competitive with nuclear fission and other low-emissions power. To start, the price of sending a tonne of cargo to LEO on a large reusable rocket, which has fallen 36 percent over the past 10 years, would have to drop by another two-thirds, to $500,000. This assumes that all the pieces of the station could be dropped off in low orbit and then raised to GEO over a period of months by space tugs propelled by electrical ion thrusters rather than conventional rockets. The approach would slow the pace of construction and add to the overall mass and cost. New tugs would have to be developed that could tow up to 100 times as much cargo as the biggest electric tugs do today. And by my calculations, the world’s annual production of xenon—the go-to propellant for ion engines—is insufficient to carry even a single solar-power satellite to GEO.

Thales Alenia Space looked at a slightly more realistic option: using a fleet of conventional rockets as big as SpaceX’s new Starship—the largest rocket ever built—to ferry loads from LEO to GEO, and then back to LEO for refueling from an orbiting fuel depot. Even if launch prices plummeted to $200,000 a tonne, they calculated, electricity from their system would be six times as expensive as NASA’s projected cost for a terrestrial solar farm outfitted with battery storage—one obvious alternative.

What else would have to go spectacularly right? In NASA’s cost-competitive scenario, the price of new, specialized spaceships that could maintain the satellite for 30 years—and then disassemble and dispose of it—would have to come down by 90 percent. The efficiency of commercially produced, space-qualified solar cells would have to soar from 32 percent today to 40 percent, while falling in cost. Yet over the past 30 years, big gains in the efficiency of research cells have not translated well to the commercial cells available at low cost [see chart, “Not So Fast”].

Is it possible for all these things to go right simultaneously? Perhaps. But wait—there’s more that can go wrong.

The Toll of Operating a Solar Plant in Space

Let’s start with temperature. Gigawatts of power coursing through the system will make heat removal essential because solar cells lose efficiency and microcircuits fry when they get too hot. A couple of dozen times a year, the satellite will pass suddenly into the utter darkness of Earth’s shadow, causing temperatures to swing by around 300 °C, well beyond the usual operating range of electronics. Thermal expansion and contraction may cause large structures on the station to warp or vibrate.

Then there’s the physical toll of operating in space. Vibrations and torques exerted by altitude-control thrusters, plus the pressure of solar radiation on the massive sail-like arrays, will continually bend and twist the station this way and that. The sprawling arrays will suffer unavoidable strikes from man-made debris and micrometeorites, perhaps even a malfunctioning construction robot. As the number of space power stations increases, we could see a rapid rise in the threat of Kessler syndrome, a runaway cascade of collisions that is every space operator’s nightmare.

Probably the toughest technical obstacle blocking space solar power is a basic one: shaping and aiming the beam. The transmitter is not a dish, like a radio telescope in reverse. It’s a phased array, a collection of millions of little antennas that must work in near-perfect synchrony, each contributing its piece to a collective waveform aimed at the ground station.

Like people in a stadium crowd raising their arms on cue to do “the wave,” coordination of a phased array is essential. It will work properly only if every element on the emitter syncs the phase of its transmission to align precisely with the transmission of its neighbors and with an incoming beacon signal sent from the ground station. Phase errors measured in picoseconds can cause the microwave beam to blur or drift off its target. How can the system synchronize elements separated by as much as a kilometer with such incredible accuracy? If you have the answer, please patent and publish it, because this problem currently has engineers stumped.

There is no denying the beauty of the idea of turning to deep space for inexhaustible electricity. But nature gets a vote. As Lao Tzu observed long ago in the Tao Te Ching, “The truth is not always beautiful, nor beautiful words the truth.”

Solar panels are built to last 25 years or more in all kinds of weather. Key to this longevity is a tight seal of the photovoltaic materials. Manufacturers achieve the seal by laminating a panel’s silicon cells with polymer sheets between glass panes. But the sticky polymer is hard to separate from the silicon cells at the end of a solar panel’s life, making recycling the materials more difficult.

Researchers at the U.S. National Renewable Energy Lab (NREL) in Golden, Colo., say they’ve found a better way to seal solar modules. Using a femtosecond laser, the researchers welded together solar panel glass without the use of polymers such as ethylene vinyl acetate. These glass-to-glass precision welds are strong enough for outdoor solar panels, and are better at keeping out corrosive moisture, the researchers say.

A femtosecond laser welds a small piece of test glass.NREL

“Solar panels are not easily recycled,” says David Young, a senior scientist at NREL. “There are companies that are doing it now, but it’s a tricky play between cost and benefit, and most of the problem is with the polymers.” With no adhesive polymers involved, recycling facilities can more easily separate and reuse the valuable materials in solar panels such as silicon, silver, copper, and glass.

Because of the polymer problem, many recycling facilities just trash the polymer-covered silicon cells and recover only the aluminum frames and glass encasements, says Silvana Ovaitt, a photovoltaic (PV) analyst at NREL. This partial recycling wastes the most valuable materials in the modules.

“At some point there’s going to be a huge amount of spent panels out there, and we want to get it right, and make it easy to recycle.” —David Young, NREL

Finding cost-effective ways to recycle all the materials in solar panels will become increasingly important. Manufacturers globally are deploying enough solar panels to produce an additional 240 gigawatts each year. That rate is projected to increase to 3 terawatts by 2030, Ovaitt says. By 2050, anywhere from 54 to 160 million tonnes of PV panels will have reached the end of their life-spans, she says.

“At some point there’s going to be a huge amount of spent panels out there, and we want to get it right, and make it easy to recycle,” says Young. “There’s no reason not to.” A change in manufacturing could help alleviate the problem—although not for at least another 25 years, when panels constructed with the new technique would be due to be retired.

In NREL’s technique, the glass that encases the solar cells in a PV panel is welded together by precision melting. The precision melting is accomplished with femtosecond lasers, which pack a tremendous number of photons into a very short time scale--about 1 millionth of 1 billionth of a second. The number of photons emitted per second from the laser is so intense that it changes the optical absorption process in the glass, says Young. The process changes from linear (normal absorption) to nonlinear, which allows the glass to absorb energy from the photons that it would normally not absorb, he says.

The intense beam, focused near the interface of the two sheets of glass, generates a small plasma of ionized glass atoms. This plasma allows the glass to absorb most of the photons from the laser and locally melt the two glass sheets to form a weld. Because there’s no open surface, there is no evaporation of the molten glass during the welding process. The lack of evaporation from the molten pool allows the glass to cool in a stress-free state, leaving a very strong weld.

A femtosecond laser creates precision welds between two glass plates.David Young/NREL

In stress tests conducted by the NREL group, the welds proved almost as strong as the glass itself, as if there were no weld at all. Young and his colleagues described their proof-of-concept technique in a paper published 21 February in the IEEE Journal of Photovoltaics.

This is the first time a femtosecond laser has been used to test glass-to-glass welds for solar modules, the authors say. The cost of such lasers has declined over the last few years, so researchers are finding uses for them in a wide range of applications. For example, femtosecond lasers have been used to create 3D midair plasma displays and to turn tellurite glass into a semiconductor crystal. They’ve also been used to weld glass in medical devices.

Prior to femtosecond lasers, research groups attempted to weld solar panel glass with nanosecond lasers. But those lasers, with pulses a million times as long as those of a femtosecond laser, couldn’t create a glass-to-glass weld. Researchers tried using a filler material called glass frit in the weld, but the bonds of the dissimilar materials proved too brittle and weak for outdoor solar panel designs, Young says.

In addition to making recycling easier, NREL’s design may make solar panels last longer. Polymers are poor barriers to moisture compared with glass, and the material degrades over time. This lets moisture into the solar cells, and eventually leads to corrosion. “Current solar modules aren’t watertight,” says Young. That will be a problem for perovskite cells, a leading next-generation solar technology that is extremely sensitive to moisture and oxygen.

“If we can provide a different kind of seal where we can eliminate the polymers, not only do we get a better module that lasts longer, but also one that is much easier to recycle,” says Young.

Today Dresden, Germany–based startup SpiNNcloud Systems announced that its hybrid supercomputing platform, the SpiNNcloud Platform, is available for sale. The machine combines traditional AI accelerators with neuromorphic computing capabilities, using system-design strategies that draw inspiration from the human brain. Systems for purchase vary in size, but the largest commercially available machine can simulate 10 billion neurons, about one-tenth the number in the human brain. The announcement was made at the ISC High Performance conference in Hamburg, Germany.

“We’re basically trying to bridge the gap between brain inspiration and artificial systems.” —Hector Gonzalez, SpiNNcloud Systems

SpiNNcloud Systems was founded in 2021 as a spin-off of the Dresden University of Technology. Its original chip, the SpiNNaker1, was designed by Steve Furber, the principal designer of the ARM microprocessor—the technology that now powers most cellphones. The SpiNNaker1 chip is already in use by 60 research groups in 23 countries, SpiNNcloud Systems says.

Human Brain as Supercomputer

Brain-emulating computers hold the promise of vastly lower energy computation and better performance on certain tasks. “The human brain is the most advanced supercomputer in the universe, and it consumes only 20 watts to achieve things that artificial intelligence systems today only dream of,” says Hector Gonzalez, cofounder and co-CEO of SpiNNcloud Systems. “We’re basically trying to bridge the gap between brain inspiration and artificial systems.”

Aside from sheer size, a distinguishing feature of the SpiNNaker2 system is its flexibility. Traditionally, most neuromorphic computers emulate the brain’s spiking nature: Neurons fire off electrical spikes to communicate with the neurons around them. The actual mechanism of these spikes in the brain is quite complex, and neuromorphic hardware often implements a specific simplified model. The SpiNNaker2 can implement a broad range of such models however, as they are not hardwired into its architecture.

Instead of looking how each neuron and synapse operates in the brain and trying to emulate that from the bottom up, Gonzalez says, the his team’s approach involved implementing key performance features of the brain. “It’s more about taking a practical inspiration from the brain, following particularly fascinating aspects such as how the brain is energy proportional and how it is simply highly parallel,” Gonzalez says.

To build hardware that is energy proportional—each piece draws power only when it’s actively in use and highly parallel—the company started with the building blocks. The basic unit of the system is the SpiNNaker2 chip, which hosts 152 processing units. Each processing unit has an ARM-based microcontroller, and unlike its predecessor the SpiNNaker1, also comes equipped with accelerators for use on neuromorphic models and traditional neural networks.

The SpiNNaker2 supercomputer has been designed to model up to 10 billion neurons, about one-tenth the number in the human brain. SpiNNCloud Systems

The processing units can operate in an event-based manner: They can stay off unless an event triggers them to turn on and operate. This enables energy-proportional operation. The events are routed between units and across chips asynchronously, meaning there is no central clock coordinating their movements—which can allow for massive parallelism. Each chip is connected to six other chips, and the whole system is connected in the shape of a torus to ensure all connecting wires are equally short.

The largest commercially offered system is not only capable of emulating 10 billion neurons, but also performing 0.3 billion billion operations per second (exaops) of more traditional AI tasks, putting it on a comparable scale with the top 10 largest supercomputers today.

Among the first customers of the SpiNNaker2 system is a team at Sandia National Labs, which plans to use it for further research on neuromorphic systems outperforming traditional architectures and performing otherwise inaccessible computational tasks.

“The ability to have a general programmable neuron model lets you explore some of these more complex learning rules that don’t necessarily fit onto older neuromorphic systems,” says Fred Rothganger, senior member of technical staff at Sandia. “They, of course, can run on a general-purpose computer. But those general-purpose computers are not necessarily designed to efficiently handle the kind of communication patterns that go on inside a spiking neural network. With [the SpiNNaker2 system] we get the ideal combination of greater programmability plus efficient communication.”

A quick glance at the news headlines each morning might convey that the world is in crisis. Challenges include climate-change threats to human infrastructure and habitats; cyberwarfare by state and nonstate actors attacking energy sources and health care systems; and the global water crisis, which is compounded by the climate crisis. Perhaps the biggest challenge is the rapid advance of artificial intelligence and what it means for humanity.

As people grapple with those and other issues, they typically look to policymakers and business leaders for answers. However, no true solutions can be developed and implemented without the technical expertise of engineers.

Encouraging visionary, futuristic thinking is the function of the Engineering Research Visioning Alliance. ERVA is an initiative of the U.S. National Science Foundation’s Directorate for Engineering, for which I serve as principal investigator. IEEE is one of several professional engineering societies that are affiliate partners.

Engineers are indispensable architects

Engineers are not simply crucial problem-solvers; they have long proven to be proactive architects of the future. For example, Nobel-winning physicists discovered the science behind the sensors that make modern photography possible. Engineers ran with the discovery, developing technology that NASA could use to send back clear pictures from space, giving us glimpses of universes far beyond our line of sight. The same tech enables you to snap photos with your cellphone.

As an engineer myself, I am proud of our history of not just making change but also envisioning it.

In the late 19th century, electrical engineer Nikola Tesla had envisioned wireless communication, lighting, and power distribution.

As early as 1900, civil engineer John Elfreth Watkins predicted that by 2000 we would have such now-commonplace innovations as color photography, wireless telephones, and home televisions (and even TV dinners), among other things.

“If we are going to successfully tackle today’s most vexing global challenges, engineers cannot be relegated to playing a reactive role.”

Watkins embodied an engineer’s curiosity and prescience, but too often today, we spend the lion’s share of our time with technical tinkering and not enough on the bigger picture.

If we are going to successfully tackle today’s most vexing global challenges, engineers cannot be relegated to playing a reactive role. We need to completely reimagine how nearly everything works. And because complex problems are multifaceted, we must do so in a multidisciplinary fashion.

We need big ideas, future-focused thinking with the foresight to transform how we live, work, and play—a visionary mindset embraced and advanced by engineers who leverage R&D to solve problems and activate discoveries. We need a different attitude from that of the consummate practitioners we typically imagine ourselves to be. We need the mindset of the futurist.

Futuristic thinking transforms society

A futurist studies current events and trends to determine not just predictions but also possibilities for the future. The term futurist has a long connection with science fiction, going back to the early 20th century, personified in such figures as writer H.G. Wells.

While many literary figures’ predictions have proven fanciful (though some, like Elfreth’s, have come true), engineers and scientists have engaged in foresight for generations, introducing new ways to look at our world, and transforming society along the way.

Futuristic thinking pushes the boundaries of what we can currently imagine and conceive. In an era of systemic crises, there is a seemingly paradoxical but accurate truth: It has become impractical to think too pragmatically.

It is especially counterintuitive to engineers, as we are biased toward observable, systematic thinking. But it is a limitation we have overcome through visionary exploits of the past—and one we must overcome now, when the world needs us.

Overcoming systematic thinking

Four times each year, ERVA convenes engineers, scientists, technologists, ethicists, social scientists, and federal science program leads to engage in innovative visioning workshops. We push hard and ask the experts to expand their thinking beyond short-term problems and think big about future possibilities. Some examples of challenges we have addressed—and the subsequent comprehensive reports on recommended research direction for visionary, futuristic thinking—are:

• The Role of Engineering to Address Climate Change. Our first visioning event considered how engineers can help mitigate the effects of rising global temperatures and better reduce carbon emissions. We envisioned how we could use artificial intelligence and other new technologies, including some revolutionary sensors, to proactively assess weather and water security events, decarbonize without disruptions to our energy supply, and slow the pace of warming.
• Engineering R&D Solutions for Unhackable Infrastructure. We considered a future in which humans and computing systems were connected using trustworthy systems, with engineering solutions to self-identity threats and secure systems before they become compromised. Solutions for unhackable infrastructure should be inherent rather than bolted-on, integrated across connected channels, and activated from the system level to wearables. Actions must be taken now to ensure trustworthiness at every level so that the human element is at the forefront of future information infrastructure.
• Engineering Materials for a Sustainable Future. In our most recent report, we discussed a future in which the most ubiquitous, noncircular materials in our world—concrete, chemicals, and single-use packaging—are created using sustainable materials. We embraced the use of organic and reusable materials, examining what it is likely to take to shift production, storage, and transportation in the process. Again, engineers are required to move beyond current solutions and to push the boundaries of what is possible.

ERVA is tackling new topics in upcoming visioning sessions on areas as diverse as the future of wireless competitiveness, quantum engineering, and improving women’s health.

We have an open call for new visioning event ideas. We challenge the engineering community to propose themes for ERVA to explore so we can create a road map of future research priorities to solve societal challenges. Engineers are needed to share their expertise, so visit our website to follow this critical work. It is time we recaptured that futurist spirit.

The emergence of artificial general intelligence (AGI)—systems that can perform any task a human can—could be the most important event in human history, one that radically affects all aspects of our collective lives. Yet AGI, which could emerge soon, remains an elusive and controversial concept. We lack a clear definition of what it is, we don’t know how we will detect it, and we don’t know how to deal with it if it finally emerges.

What we do know, however, is that today’s approaches to studying AGI are not nearly rigorous enough. Within industry, where many of today’s AI breakthroughs are happening, companies like OpenAI are actively striving to create AGI, but include research on AGI’s social dimensions and safety issues only as their corporate leaders see fit. While the academic community looks at AGI more broadly, seeking the characteristics of a new intelligent life-form, academic institutions don’t have the resources for a significant effort.

Thinking about AGI calls to mind another poorly understood and speculative phenomenon with the potential for transformative impacts on humankind. We believe that the SETI Institute’s efforts to detect advanced extraterrestrial intelligence demonstrate several valuable concepts that can be adapted for AGI research. Instead of taking a dogmatic or sensationalist stance, the SETI project takes a scientifically rigorous and pragmatic approach—putting the best possible mechanisms in place for the definition, detection, and interpretation of signs of possible alien intelligence.

The idea behind SETI goes back 60 years, to the beginning of the space age. In their 1959 Nature paper, the physicists Giuseppe Cocconi and Philip Morrison described the need to search for interstellar communication. Assuming the uncertainty of extraterrestrial civilizations’ existence and technological sophistication, they theorized about how an alien society would try to communicate and discussed how we should best “listen” for messages. Inspired by this position, we argue for a similar approach to studying AGI, in all its uncertainties.

AI researchers are still debating how probable it is that AGI will emerge and how to detect it. However, the challenges in defining AGI and the difficulties in measuring it are not a justification for ignoring it or for taking a “we’ll know when we see it” approach. On the contrary, these issues strengthen the need for an interdisciplinary approach to AGI detection, evaluation, and public education, including a science-based approach to the risks associated with AGI.

We need a SETI-like approach to AGI now

The last few years have shown a vast leap in AI capabilities. The large language models (LLMs) that power chatbots like ChatGPT, which can converse convincingly with humans, have renewed the discussion about AGI. For example, recent articles have stated that ChatGPT shows “sparks” of AGI, is capable of reasoning, and outperforms humans in many evaluations.

While these claims are intriguing and exciting, there are reasons to be skeptical. In fact, a large group of scientists argue that the current set of tools won’t bring us any closer to true AGI. But given the risks associated with AGI, if there is even a small likelihood of it occurring, we must make a serious effort to develop a standard definition of AGI, establish a SETI-like approach to detecting it, and devise ways to safely interact with it if it emerges.

Challenge 1: How to define AGI

The crucial first step is to define what exactly to look for. In SETI’s case, researchers decided to look for so-called narrow-band signals distinct from other radio signals present in the cosmic background. These signals are considered intentional and only produced by intelligent life.

In the case of AGI, matters are far more complicated. Today, there is no clear definition of “artificial general intelligence” (other terms, such as strong AI, human-level intelligence, and superintelligence are also widely used to describe similar concepts). The term is hard to define because it contains other imprecise and controversial terms. Although “intelligence” is defined in the Oxford Dictionary as “the ability to acquire and apply knowledge and skills,” there is still much debate on which skills are involved and how they can be measured. The term “general” is also ambiguous. Does an AGI need to be able to do everything a human can do? Is generality a quality we measure as a binary or continuous variable?

One of the first missions of a “SETI for AGI” construct must be to clearly define the terms “general” and “intelligence” so the research community can speak about them concretely and consistently. These definitions need to be grounded in the disciplines supporting the AGI concept, such as computer science, measurement science, neuroscience, psychology, mathematics, engineering, and philosophy. Once we have clear definitions of these terms, we’ll need to find ways to measure them.

There’s also the crucial question of whether a true AGI must include consciousness, personhood, and self-awareness. These terms also have multiple definitions, and the relationships between them and intelligence must be clarified. Although it’s generally thought that consciousness isn’t necessary for intelligence, it’s often intertwined with discussions of AGI because creating a self-aware machine would have many philosophical, societal, and legal implications. Would a new large language model that can answer an IQ test better than a human be as important to detect as a truly conscious machine?

Getty Images

Challenge 2: How to measure AGI

In the case of SETI, if a candidate narrow-band signal is detected, an expert group will verify that it is indeed an extraterrestrial source. They’ll use established criteria—for example, looking at the signal type and source and checking for repetition—and conduct all the assessments at multiple facilities for additional validation.

How to best measure computer intelligence has been a long-standing question in the field. In a famous 1950 paper, Alan Turing proposed the “imitation game,” now more widely known as the Turing Test, which assesses whether human interlocutors can distinguish if they are chatting with a human or a machine. Although the Turing Test has been useful for evaluations in the past, the rise of LLMs has made it clear that it’s not a complete enough test to measure intelligence. As Turing noted in his paper, the imitation game does an excellent job of testing if a computer can imitate the language-generation process, but the relationship between imitating language and thinking is still an open question. Other techniques will certainly be needed.

These appraisals must be directed at different dimensions of intelligence. Although measures of human intelligence are controversial, IQ tests can provide an initial baseline to assess one dimension. In addition, cognitive tests on topics such as creative problem-solving, rapid learning and adaptation, reasoning, goal-directed behavior, and self-awareness would be required to assess the general intelligence of a system.

These cognitive tests will be useful, but it’s important to remember that they were designed for humans and might contain certain assumptions about basic human capabilities that might not apply to computers, even those with AGI abilities. For example, depending on how it’s trained, a machine may score very high on an IQ test but remain unable to solve much simpler tasks. In addition, the AI may have other communication modalities and abilities that would not be measurable by our traditional tests.

There’s a clear need to design novel evaluations to measure AGI or its subdimensions accurately. This process would also require a diverse set of researchers from different fields who deeply understand AI, are familiar with the currently available tests, and have the competency, creativity, and foresight to design novel tests. These measurements will hopefully alert us when meaningful progress is made toward AGI.

Once we have developed a standard definition of AGI and developed methodologies to detect it, we must devise a way to address its emergence.

Challenge 3: How to deal with AGI

Once we have discovered this new form of intelligence, we must be prepared to answer questions such as: Is the newly discovered intelligence a new form of life? What kinds of rights does it have? What kinds of rights do we have regarding this intelligence? What are the potential safety concerns, and what is our approach to handling the AGI entity, containing it, and safeguarding ourselves from it?

Here, too, SETI provides inspiration. SETI has protocols for handling the evidence of a sign of extraterrestrial intelligence. SETI’s post-detection protocols emphasize validation, transparency, and cooperation with the United Nations, with the goal of maximizing the credibility of the process, minimizing sensationalism, and bringing structure to such a profound event.

As with extraterrestrial intelligence, we need protocols for safe and secure interactions with AGI. These AGI protocols would serve as the internationally recognized framework for validating emergent AGI properties, bringing transparency to the entire process, ensuring international cooperation, applying safety-related best practices, and handling any ethical, social, and philosophical concerns.

We readily acknowledge that the SETI analogy can only go so far. If AGI emerges, it will be a human-made phenomenon. We will likely gradually engineer AGI and see it slowly emerge, so detection might be a process that takes place over a period of years, if not decades. In contrast, the existence of extraterrestrial life is something that we have no control over, and contact could happen very suddenly.

The discovery of a true AGI would be the most profound development in the history of science, and its consequences would be also entirely unpredictable. To best prepare, we need a methodical, comprehensive, principled, and interdisciplinary approach to defining, detecting, and dealing with AGI. With SETI as an inspiration, we propose that the AGI research community establish a similar framework to ensure an unbiased, scientific, transparent, and collaborative approach to dealing with possibly the most important development in human history.

The IEEE–Eta Kappa Nu honor society for engineers celebrates its 120th anniversary this year. Founded in October 1904, IEEE-HKN recognizes academic experience as well as excellence in scholarship, leadership, and service. Inductees are chosen based on their technical, scientific, and leadership achievements. There are now more than 270 IEEE-HKN chapters at universities around the world.

The society has changed significantly over the years. Global expansion resulted from the merger of North America–based HKN with IEEE in 2010. There are now 30 chapters outside the United States, including ones recently established at universities in Ecuador, Hungary, and India.

IEEE-HKN has more than 200,000 members around the world. Since the merger, more than 37,000 people have been inducted. Membership now extends beyond just students. Among them are 23 former IEEE presidents as well as a who’s who of engineering leaders and technology pioneers including GM Chief Executive Mary Barra, Google founding CEO Larry Page, and Advanced Micro Devices CEO Lisa Su. Last year more than 100 professional members were added to the rolls.

“If you want to make sure that you’re on the forefront of engineering leadership, you should definitely consider joining IEEE-HKN.” —Joseph Greene

In 1950 HKN established the category of eminent member to honor those whose contributions significantly benefited society. There now are 150 such members. They include the fathers of the Internet and IEEE Medal of Honor recipients Vint Cerf and Bob Kahn; former astronaut Sandra Magnus; and Henry Samueli, a Broadcom founder.

IEEE-HKN is celebrating its anniversary on 28 October, Founders Day, the date the society was established. A variety of activities are scheduled for the day at chapters and other locations around the world, says Nancy Ostin, the society’s director.

New chapters in Ecuador, Hungary, and India

Several chapters have been established in recent months. The Nu Eta chapter at the Sri Sairam Engineering College, in Chennai, India, was founded in September, becoming the fourth chapter in the country. In October the Nu Theta chapter debuted at Purdue University Northwest in Hammond, Ind.

Students from the IEEE-HKN Lambda Chi chapter at Hampton University in Virginia celebrate their induction with a cake. IEEE-Eta Kappa Nu

So far this year, chapters were formed at the Escuela Superior Politécnica del Litoral, in Guayaquil, Ecuador; Hampton University in Virginia; Óbuda University, in Budapest; and Polytechnic University of Puerto Rico in San Juan, the second chapter in the territory. Hampton is a historically Black research university.

A focus on career development

IEEE-HKN’s benefits have expanded over time. The society now focuses more on helping its members with career development. Career-related services on the society’s website include a job board and a resource center that aids with writing résumés and cover letters, as well as interview tips and career coaching services.

2024 IEEE-HKN president Ryan Bales [center] with members of the Nu Iota chapter at Óbuda University in Budapest. IEEE-Eta Kappa Nu

There’s also the HKN Career Conversations podcast, hosted by society alumni. Topics they’ve covered include ethics, workplace conflicts, imposter syndrome, and cultivating creativity.

The honor society also holds networking events including its annual international leadership conferences, where student leaders from across the world collaborate on how they can benefit the organization and their communities.

Mentorship and networking opportunities

IEEE-HKN’s mentoring program connects recent graduates with alumni. IEEE professionals are paired with graduate students based on technical interest, desired mentoring area, and personality.

Alumnus Joseph Greene, a Ph.D. candidate in computational imaging at Boston University, joined the school’s Kappa Sigma chapter in 2014 and continues to mentor graduate students and help organize events to engage alumni. Greene has held several leadership positions with the chapter, including president, vice president, and student governor on the IEEE-HKN board.

He created a professional-to-student mentoring program for the chapter. It partners people from industry and academia with students to build working relationships and to provide career, technical, and personal advice. Since the program launched in 2022, Greene says, more than 40 people have participated.

“What I found most rewarding about having a mentor is they offer a much broader perspective than just your collegiate needs,” he said in the interview with The Institute.

Another program Greene launched is the IEEE-HKN GradLab YouTube podcast, which he says covers “everything about grad school that they don’t teach you in a classroom.”

“If you want to make sure that you’re on the forefront of engineering leadership, you should definitely consider joining IEEE-HKN,” Greene said in the interview. “The organization, staff, and volunteers are dedicated toward making sure you have the opportunity, resources, and network to thrive and succeed.”

If you were ever inducted into IEEE-HKN, your membership never expires, Ostin notes. Check your IEEE membership record. The honor society’s name should appear there but if it does not, complete the alumni reconnect form.

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Eurobot Open 2024: 8–11 May 2024, LA ROCHE-SUR-YON, FRANCE

ICRA 2024: 13–17 May 2024, YOKOHAMA, JAPAN

RoboCup 2024: 17–22 July 2024, EINDHOVEN, NETHERLANDS

Cybathlon 2024: 25–27 October 2024, ZURICH

Enjoy today’s videos!

In this work, we present LocoMan, a dexterous quadrupedal robot with a novel morphology to perform versatile manipulation in diverse constrained environments. By equipping a Unitree Go1 robot with two low-cost and lightweight modular 3-DoF loco-manipulators on its front calves, LocoMan leverages the combined mobility and functionality of the legs and grippers for complex manipulation tasks that require precise 6D positioning of the end effector in a wide workspace.

[ CMU ]

Thanks, Changyi!

Object manipulation has been extensively studied in the context of fixed-base and mobile manipulators. However, the overactuated locomotion modality employed by snake robots allows for a unique blend of object manipulation through locomotion, referred to as loco-manipulation. In this paper, we present an optimization approach to solving the loco-manipulation problem based on nonimpulsive implicit-contact path planning for our snake robot COBRA.

[ Silicon Synapse Lab ]

Okay, but where that costume has eyes is not where Spot has eyes, so the Spot in the costume can’t see, right? And now I’m skeptical of the authenticity of the mutual snoot-boop.

[ Boston Dynamics ]

Here’s some video of Field AI’s robots operating in relatively complex and unstructured environments without prior maps. Make sure to read our article from this week for details!

[ Field AI ]

Is it just me, or is it kind of wild that researchers are now publishing papers comparing their humanoid controller to the “manufacturer’s” humanoid controller? It’s like humanoids are a commodity now or something.

[ OSU ]

I, too, am packing armor for ICRA.

[ Pollen Robotics ]

Honey Badger 4.0 is our latest robotic platform, created specifically for traversing hostile environments and difficult terrains. Equipped with multiple cameras and sensors, it will make sure no defect is omitted during inspection.

[ MAB Robotics ]

Thanks, Jakub!

Have an automation task that calls for the precision and torque of an industrial robot arm…but you need something that is more rugged or a nonconventional form factor? Meet the HEBI Robotics H-Series Actuator! With 9x the torque of our X-Series and seamless compatibility with the HEBI ecosystem for robot development, the H-Series opens a new world of possibilities for robots.

[ HEBI ]

Thanks, Dave!

This is how all spills happen at my house too: super passive-aggressively.

[ 1X ]

EPFL’s team, led by Ph.D. student Milad Shafiee along with coauthors Guillaume Bellegarda and BioRobotics Lab head Auke Ijspeert, have trained a four-legged robot using deep-reinforcement learning to navigate challenging terrain, achieving a milestone in both robotics and biology.

[ EPFL ]

At Agility, we make robots that are made for work. Our robot Digit works alongside us in spaces designed for people. Digit handles the tedious and repetitive tasks meant for a machine, allowing companies and their people to focus on the work that requires the human element.

[ Agility ]

With a wealth of incredible figures and outstanding facts, here’s Jan Jonsson, ABB Robotics veteran, sharing his knowledge and passion for some of our robots and controllers from the past.

[ ABB ]

I have it on good authority that getting robots to mow a lawn (like, any lawn) is much harder than it looks, but Electric Sheep has built a business around it.

[ Electric Sheep ]

The AI Index, currently in its seventh year, tracks, collates, distills, and visualizes data relating to artificial intelligence. The Index provides unbiased, rigorously vetted, and globally sourced data for policymakers, researchers, journalists, executives, and the general public to develop a deeper understanding of the complex field of AI. Led by a steering committee of influential AI thought leaders, the Index is the world’s most comprehensive report on trends in AI. In this seminar, HAI Research Manager Nestor Maslej offers highlights from the 2024 report, explaining trends related to research and development, technical performance, technical AI ethics, the economy, education, policy and governance, diversity, and public opinion.

[ Stanford HAI ]

This week’s CMU Robotics Institute seminar, from Dieter Fox at Nvidia and the University of Washington, is “Where’s RobotGPT?”

In this talk, I will discuss approaches to generating large datasets for training robot-manipulation capabilities, with a focus on the role simulation can play in this context. I will show some of our prior work, where we demonstrated robust sim-to-real transfer of manipulation skills trained in simulation, and then present a path toward generating large-scale demonstration sets that could help train robust, open-world robot-manipulation models.

[ CMU ]

Dina Genkina: Hi, I’m Dina Genkina for IEEE Spectrum‘s Fixing the Future. Before we start, I want to tell you that you can get the latest coverage from some of Spectrum‘s most important beats, including AI, climate change, and robotics, by signing up for one of our free newsletters. Just go to spectrum.ieee.org/newsletters to subscribe. And today our guest on the show is Suraj Bramhavar. Recently, Bramhavar left his job as a co-founder and CTO of Sync Computing to start a new chapter. The UK government has just founded the Advanced Research Invention Agency, or ARIA, modeled after the US’s own DARPA funding agency. Bramhavar is heading up ARIA’s first program, which officially launched on March 12th of this year. Bramhavar’s program aims to develop new technology to make AI computation 1,000 times more cost efficient than it is today. Siraj, welcome to the show.

Suraj Bramhavar: Thanks for having me.

Genkina: So your program wants to reduce AI training costs by a factor of 1,000, which is pretty ambitious. Why did you choose to focus on this problem?

Bramhavar: So there’s a couple of reasons why. The first one is economical. I mean, AI is basically to become the primary economic driver of the entire computing industry. And to train a modern large-scale AI model costs somewhere between 10 million to 100 million pounds now. And AI is really unique in the sense that the capabilities grow with more computing power thrown at the problem. So there’s kind of no sign of those costs coming down anytime in the future. And so this has a number of knock-on effects. If I’m a world-class AI researcher, I basically have to choose whether I go work for a very large tech company that has the compute resources available for me to do my work or go raise 100 million pounds from some investor to be able to do cutting edge research. And this has a variety of effects. It dictates, first off, who gets to do the work and also what types of problems get addressed. So that’s the economic problem. And then separately, there’s a technological one, which is that all of this stuff that we call AI is built upon a very, very narrow set of algorithms and an even narrower set of hardware. And this has scaled phenomenally well. And we can probably continue to scale along kind of the known trajectories that we have. But it’s starting to show signs of strain. Like I just mentioned, there’s an economic strain, there’s an energy cost to all this. There’s logistical supply chain constraints. And we’re seeing this now with kind of the GPU crunch that you read about in the news.

And in some ways, the strength of the existing paradigm has kind of forced us to overlook a lot of possible alternative mechanisms that we could use to kind of perform similar computations. And this program is designed to kind of shine a light on those alternatives.

Genkina: Yeah, cool. So you seem to think that there’s potential for pretty impactful alternatives that are orders of magnitude better than what we have. So maybe we can dive into some specific ideas of what those are. And you talk about in your thesis that you wrote up for the start of this program, you talk about natural computing systems. So computing systems that take some inspiration from nature. So can you explain a little bit what you mean by that and what some of the examples of that are?

Bramhavar: Yeah. So when I say natural-based or nature-based computing, what I really mean is any computing system that either takes inspiration from nature to perform the computation or utilizes physics in a new and exciting way to perform computation. So you can think about kind of people have heard about neuromorphic computing. Neuromorphic computing fits into this category, right? It takes inspiration from nature and usually performs a computation in most cases using digital logic. But that represents a really small slice of the overall breadth of technologies that incorporate nature. And part of what we want to do is highlight some of those other possible technologies. So what do I mean when I say nature-based computing? I think we have a solicitation call out right now, which calls out a few things that we’re interested in. Things like new types of in-memory computing architectures, rethinking AI models from an energy context. And we also call out a couple of technologies that are pivotal for the overall system to function, but are not necessarily so eye-catching, like how you interconnect chips together, and how you simulate a large-scale system of any novel technology outside of the digital landscape. I think these are critical pieces to realizing the overall program goals. And we want to put some funding towards kind of boosting that workup as well.

Genkina: Okay, so you mentioned neuromorphic computing is a small part of the landscape that you’re aiming to explore here. But maybe let’s start with that. People may have heard of neuromorphic computing, but might not know exactly what it is. So can you give us the elevator pitch of neuromorphic computing?

Bramhavar: Yeah, my translation of neuromorphic computing— and this may differ from person to person, but my translation of it is when you kind of encode the information in a neural network via spikes rather than kind of discrete values. And that modality has shown to work pretty well in certain situations. So if I have some camera and I need a neural network next to that camera that can recognize an image with very, very low power or very, very high speed, neuromorphic systems have shown to work remarkably well. And they’ve worked in a variety of other applications as well. One of the things that I haven’t seen, or maybe one of the drawbacks of that technology that I think I would love to see someone solve for is being able to use that modality to train large-scale neural networks. So if people have ideas on how to use neuromorphic systems to train models at commercially relevant scales, we would love to hear about them and that they should submit to this program call, which is out.

Genkina: Is there a reason to expect that these kinds of— that neuromorphic computing might be a platform that promises these orders of magnitude cost improvements?

Bramhavar: I don’t know. I mean, I don’t know actually if neuromorphic computing is the right technological direction to realize that these types of orders of magnitude cost improvements. It might be, but I think we’ve intentionally kind of designed the program to encompass more than just that particular technological slice of the pie, in part because it’s entirely possible that that is not the right direction to go. And there are other more fruitful directions to put funding towards. Part of what we’re thinking about when we’re designing these programs is we don’t really want to be prescriptive about a specific technology, be it neuromorphic computing or probabilistic computing or any particular thing that has a name that you can attach to it. Part of what we tried to do is set a very specific goal or a problem that we want to solve. Put out a funding call and let the community kind of tell us which technologies they think can best meet that goal. And that’s the way we’ve been trying to operate with this program specifically. So there are particular technologies we’re kind of intrigued by, but I don’t think we have any one of them selected as like kind of this is the path forward.

Genkina: Cool. Yeah, so you’re kind of trying to see what architecture needs to happen to make computers as efficient as brains or closer to the brain’s efficiency.

Bramhavar: And you kind of see this happening in the AI algorithms world. As these models get bigger and bigger and grow their capabilities, they’re starting to introduce things that we see in nature all the time. I think probably the most relevant example is this stable diffusion, this neural network model where you can type in text and generate an image. It’s got diffusion in the name. Diffusion is a natural process. Noise is a core element of this algorithm. And so there’s lots of examples like this where they’ve kind of— that community is taking bits and pieces or inspiration from nature and implementing it into these artificial neural networks. But in doing that, they’re doing it incredibly inefficiently.

Genkina: Yeah. Okay, so great. So the idea is to take some of the efficiencies out in nature and kind of bring them into our technology. And I know you said you’re not prescribing any particular solution and you just want that general idea. But nevertheless, let’s talk about some particular solutions that have been worked on in the past because you’re not starting from zero and there are some ideas about how to do this. So I guess neuromorphic computing is one such idea. Another is this noise-based computing, something like probabilistic computing. Can you explain what that is?

Bramhavar: Noise is a very intriguing property? And there’s kind of two ways I’m thinking about noise. One is just how do we deal with it? When you’re designing a digital computer, you’re effectively designing noise out of your system, right? You’re trying to eliminate noise. And you go through great pains to do that. And as soon as you move away from digital logic into something a little bit more analog, you spend a lot of resources fighting noise. And in most cases, you eliminate any benefit that you get from your kind of newfangled technology because you have to fight this noise. But in the context of neural networks, what’s very interesting is that over time, we’ve kind of seen algorithms researchers discover that they actually didn’t need to be as precise as they thought they needed to be. You’re seeing the precision kind of come down over time. The precision requirements of these networks come down over time. And we really haven’t hit the limit there as far as I know. And so with that in mind, you start to ask the question, “Okay, how precise do we actually have to be with these types of computations to perform the computation effectively?” And if we don’t need to be as precise as we thought, can we rethink the types of hardware platforms that we use to perform the computations?

So that’s one angle is just how do we better handle noise? The other angle is how do we exploit noise? And so there’s kind of entire textbooks full of algorithms where randomness is a key feature. I’m not talking necessarily about neural networks only. I’m talking about all algorithms where randomness plays a key role. Neural networks are kind of one area where this is also important. I mean, the primary way we train neural networks is stochastic gradient descent. So noise is kind of baked in there. I talked about stable diffusion models like that where noise becomes a key central element. In almost all of these cases, all of these algorithms, noise is kind of implemented using some digital random number generator. And so there the thought process would be, “Is it possible to redesign our hardware to make better use of the noise, given that we’re using noisy hardware to start with?” Notionally, there should be some savings that come from that. That presumes that the interface between whatever novel hardware you have that is creating this noise, and the hardware you have that’s performing the computing doesn’t eat away all your gains, right? I think that’s kind of the big technological roadblock that I’d be keen to see solutions for, outside of the algorithmic piece, which is just how do you make efficient use of noise.

When you’re thinking about implementing it in hardware, it becomes very, very tricky to implement it in a way where whatever gains you think you had are actually realized at the full system level. And in some ways, we want the solutions to be very, very tricky. The agency is designed to fund very high risk, high reward type of activities. And so there in some ways shouldn’t be consensus around a specific technological approach. Otherwise, somebody else would have likely funded it.

Genkina: You’re already becoming British. You said you were keen on the solution.

Bramhavar: I’ve been here long enough.

Genkina: It’s showing. Great. Okay, so we talked a little bit about neuromorphic computing. We talked a little bit about noise. And you also mentioned some alternatives to backpropagation in your thesis. So maybe first, can you explain for those that might not be familiar what backpropagation is and why it might need to be changed?

Bramhavar: Yeah, so this algorithm is essentially the bedrock of all AI training currently you use today. Essentially, what you’re doing is you have this large neural network. The neural network is composed of— you can think about it as this long chain of knobs. And you really have to tune all the knobs just right in order to get this network to perform a specific task, like when you give it an image of a cat, it says that it is a cat. And so what backpropagation allows you to do is to tune those knobs in a very, very efficient way. Starting from the end of your network, you kind of tune the knob a little bit, see if your answer gets a little bit closer to what you’d expect it to be. Use that information to then tune the knobs in the previous layer of your network and keep on doing that iteratively. And if you do this over and over again, you can eventually find all the right positions of your knobs such that your network does whatever you’re trying to do. And so this is great. Now, the issue is every time you tune one of these knobs, you’re performing this massive mathematical computation. And you’re typically doing that across many, many GPUs. And you do that just to tweak the knob a little bit. And so you have to do it over and over and over and over again to get the knobs where you need to go.

There’s a whole bevy of algorithms. What you’re really doing is kind of minimizing error between what you want the network to do and what it’s actually doing. And if you think about it along those terms, there’s a whole bevy of algorithms in the literature that kind of minimize energy or error in that way. None of them work as well as backpropagation. In some ways, the algorithm is beautiful and extraordinarily simple. And most importantly, it’s very, very well suited to be parallelized on GPUs. And I think that is part of its success. But one of the things I think both algorithmic researchers and hardware researchers fall victim to is this chicken and egg problem, right? Algorithms researchers build algorithms that work well on the hardware platforms that they have available to them. And at the same time, hardware researchers develop hardware for the existing algorithms of the day. And so one of the things we want to try to do with this program is blend those worlds and allow algorithms researchers to think about what is the field of algorithms that I could explore if I could rethink some of the bottlenecks in the hardware that I have available to me. Similarly in the opposite direction.

Genkina: Imagine that you succeeded at your goal and the program and the wider community came up with a 1/1000s compute cost architecture, both hardware and software together. What does your gut say that that would look like? Just an example. I know you don’t know what’s going to come out of this, but give us a vision.

Bramhavar: Similarly, like I said, I don’t think I can prescribe a specific technology. What I can say is that— I can say with pretty high confidence, it’s not going to just be one particular technological kind of pinch point that gets unlocked. It’s going to be a systems level thing. So there may be individual technology at the chip level or the hardware level. Those technologies then also have to meld with things at the systems level as well and the algorithms level as well. And I think all of those are going to be necessary in order to reach these goals. I’m talking kind of generally, but what I really mean is like what I said before is we got to think about new types of hardware. We also have to think about, “Okay, if we’re going to scale these things and manufacture them in large volumes cost effectively, we’re going to have to build larger systems out of building blocks of these things. So we’re going to have to think about how to stitch them together in a way that makes sense and doesn’t eat away any of the benefits. We’re also going to have to think about how to simulate the behavior of these things before we build them.” I think part of the power of the digital electronics ecosystem comes from the fact that you have cadence and synopsis and these EDA platforms that allow you with very high accuracy to predict how your circuits are going to perform before you build them. And once you get out of that ecosystem, you don’t really have that.

So I think it’s going to take all of these things in order to actually reach these goals. And I think part of what this program is designed to do is kind of change the conversation around what is possible. So by the end of this, it’s a four-year program. We want to show that there is a viable path towards this end goal. And that viable path could incorporate kind of all of these aspects of what I just mentioned.

Genkina: Okay. So the program is four years, but you don’t necessarily expect like a finished product of a 1/1000s cost computer by the end of the four years, right? You kind of just expect to develop a path towards it.

Bramhavar: Yeah. I mean, ARIA was kind of set up with this kind of decadal time horizon. We want to push out-- we want to fund, as I mentioned, high-risk, high reward technologies. We have this kind of long time horizon to think about these things. I think the program is designed around four years in order to kind of shift the window of what the world thinks is possible in that timeframe. And in the hopes that we change the conversation. Other folks will pick up this work at the end of that four years, and it will have this kind of large-scale impact on a decadal.

Genkina: Great. Well, thank you so much for coming today. Today we spoke with Dr. Suraj Bramhavar, lead of the first program headed up by the UK’s newest funding agency, ARIA. He filled us in on his plans to reduce AI costs by a factor of 1,000, and we’ll have to check back with him in a few years to see what progress has been made towards this grand vision. For IEEE Spectrum, I’m Dina Genkina, and I hope you’ll join us next time on Fixing the Future.

In 1912, Oskar von Miller, an electrical engineer and founder of the Deutsches Museum, had an idea: Could you project an artificial starry sky onto a dome, as a way of demonstrating astronomical principles to the public?

It was such a novel concept that when von Miller approached the Carl Zeiss company in Jena, Germany, to manufacture such a projector, they initially rebuffed him. Eventually, they agreed, and under the guidance of lead engineer Walther Bauersfeld, Zeiss created something amazing.

The use of models to show the movements of the planets and stars goes back centuries, starting with mechanical orreries that used clockwork mechanisms to depict our solar system. A modern upgrade was Clair Omar Musser’s desktop electric orrery, which he designed for the Seattle World’s Fair in 1962.

The projector that Zeiss planned for the Deutsches Museum would be far more elaborate. For starters, there would be two planetariums. One would showcase the Copernican, or heliocentric, sky, displaying the stars and planets as they revolved around the sun. The other would show the Ptolemaic, or geocentric, sky, with the viewer fully immersed in the view, as if standing on the surface of the Earth, seemingly at the center of the universe.

The task of realizing those ideas fell to Bauersfeld, a mechanical engineer by training and a managing director at Zeiss.

Zeiss engineer Walther Bauersfeld worked out the electromechanical details of the planetarium. In this May 1920 entry from his lab notebook [right], he sketched the two-axis system for showing the daily and annual motions of the stars.ZEISS Archive

At first, Bauersfeld focused on projecting just the sun, moon, and planets of our solar system. At the suggestion of his boss, Rudolf Straubel, he added stars. World War I interrupted the work, but by 1920 Bauersfeld was back at it. One entry in May 1920 in Bauersfeld’s meticulous lab notebook showed the earliest depiction of the two-axis design that allowed for the display of the daily as well as the annual motions of the stars. (The notebook is preserved in the Zeiss Archive.)

The planetarium projector was in fact a concatenation of many smaller projectors and a host of gears. According to the Zeiss Archive, a large sphere held all of the projectors for the fixed stars as well as a “planet cage” that held projectors for the sun, the moon, and the planets Mercury, Venus, Mars, Jupiter, and Saturn. The fixed-star sphere was positioned so that it projected outward from the exact center of the dome. The planetarium also had projectors for the Milky Way and the names of major constellations.

The projectors within the planet cage were organized in tiers with complex gearing that allowed a motorized drive to move them around one axis to simulate the annual rotations of these celestial objects against the backdrop of the stars. The entire projector could also rotate around a second axis, simulating the Earth’s polar axis, to show the rising and setting of the sun, moon, and planets over the horizon.

The Zeiss planetarium projected onto a spherical surface, which consisted of a geodesic steel lattice overlaid with concrete.Zeiss Archive

Bauersfeld also contributed to the design of the surrounding projection dome, which achieved its exactly spherical surface by way of a geodesic network of steel rods covered by a thin layer of concrete.

Planetariums catch on worldwide

The first demonstration of what became known as the Zeiss Model I projector took place on 21 October 1923 before the Deutsches Museum committee in their not-yet-completed building, in Munich. “This planetarium is a marvel,” von Miller declared in an administrative report.

In 1924, public demonstrations of the Zeiss planetarium took place on the roof of the company’s factory in Jena, Germany.ZEISS Archive

The projector then returned north to Jena for further adjustments and testing. The company also began offering demonstrations of the projector in a makeshift dome on the roof of its factory. From July to September 1924, more than 30,000 visitors experienced the Zeisshimmel (Zeiss sky) this way. These demonstrations became informal visitor-experience studies and allowed Zeiss and the museum to make refinements and improvements.

On 7 May 1925, the world’s first projection planetarium officially opened to the public at the Deutsches Museum. The Zeiss Model I displayed 4,500 stars, the band of the Milky Way, the sun, moon, Mercury, Venus, Mars, Jupiter, and Saturn. Gears and motors moved the projector to replicate the changes in the sky as Earth rotated on its axis and revolved around the sun. Visitors viewed this simulation of the night sky from the latitude of Munich and in the comfort of a climate-controlled building, although at first chairs were not provided. (I get a crick in the neck just thinking about it.) The projector was bolted to the floor, but later versions were mounted on rails to move them back and forth. A presenter operated the machine and lectured on astronomical topics, pointing out constellations and the orbits of the planets.

Word of the Zeiss planetarium spread quickly, through postcards and images.ZEISS Archive

The planetarium’s influence quickly extended far beyond Germany, as museums and schools around the world incorporated the technology into immersive experiences for science education and public outreach. Each new planetarium was greeted with curiosity and excitement. Postcards and images of planetariums (both the distinctive domed buildings and the complicated machines) circulated widely.

In 1926, Zeiss opened its own planetarium in Jena based on Bauersfeld’s specifications. The first city outside of Germany to acquire a Zeiss planetarium was Vienna. It opened in a temporary structure on 7 May 1927 and in a permanent structure four years later, only to be destroyed during World War II.

The Zeiss planetarium in Rome, which opened in 1928, projected the stars onto the domed vault of the 3rd-century Aula Ottagona, part of the ancient Baths of Diocletian.

The first planetarium in the western hemisphere opened in Chicago in May 1930. Philanthropist Max Adler, a former executive at Sears, contributed funds to the building that now bears his name. He called it a “classroom under the heavens.”

Japan’s first planetarium, a Zeiss Model II, opened in Osaka in 1937 at the Osaka City Electricity Science Museum. As its name suggests, the museum showcased exhibits on electricity, funded by the municipal power company. The city council had to be convinced of the educational value of the planetarium. But the mayor and other enthusiasts supported it. The planetarium operated for 50 years.

Who doesn’t love a planetarium?

After World War II and the division of Germany, the Zeiss company also split in two, with operations continuing at Oberkochen in the west and Jena in the east. Both branches continued to develop the planetarium through the Zeiss Model VI before shifting the nomenclature to more exotic names, such as the Spacemaster, Skymaster, and Cosmorama.

The two large spheres of the Zeiss Model II, introduced in 1926, displayed the skies of the northern and southern hemispheres, respectively. Each sphere contained a number of smaller projectors.ZEISS Archive

Over the years, refinements included increased precision, the addition of more stars, automatic controls that allowed the programming of complete shows, and a shift to fiber optics and LED lighting. Zeiss still produces planetariums in a variety of configurations for different size domes.

Today more than 4,000 planetariums are in operation globally. A planetarium is often the first place where children connect what they see in the night sky to a broader science and an understanding of the universe. My hometown of Richmond, Va., opened its first planetarium in April 1983 at the Science Museum of Virginia. That was a bit late in the big scheme of things, but just in time to wow me as a kid. I still remember the first show I saw, narrated by an animatronic Mark Twain with a focus on the 1986 visit of Halley’s Comet.

By then the museum also had a giant OmniMax screen that let me soar over the Grand Canyon, watch beavers transform the landscape, and swim with whale sharks, all from the comfort of my reclining seat. No wonder the museum is where I got my start as a public historian of science and technology. I began volunteering there at age 14 and have never looked back.

Part of a continuing series looking at historical artifacts that embrace the boundless potential of technology.

An abridged version of this article appears in the May 2024 print issue as “A Planetarium Is Born.”

References

In 2023, the Deutsches Museum celebrated the centennial of its planetarium, with the exhibition 100 Years of the Planetarium, which included artifacts such as astrolabes and armillary spheres as well as a star show in a specially built planetarium dome.

I am always appreciative of corporations that recognize their own history and maintain robust archives. Zeiss has a wonderful collection of historic photos online with detailed descriptions.

I also consulted The Zeiss Works and the Carl Zeiss Foundation in Jena by Felix Auerbach, although I read an English translation that was in the Robert B. Ariail Collection of Historical Astronomy, part of the University of South Carolina’s special collections.

The shift toward the electrification of vehicles and the expansion of the electrical grid for renewable energy integration has led to a considerable increase in the demand for power electronics devices and modernized cable systems — applications that will help ensure a consistent and long-term electricity supply. Simulation is used to drive the design of new power electronics devices (such as solar power and wind turbine systems), which are required to operate efficiently for varying levels of power production and power consumption (in the case of electric vehicles). A multiphysics modeling and simulation approach plays a critical role in meeting design goals and reducing the overall production time.

The COMSOL Multiphysics® software offers a wide range of capabilities for the modeling of energy transmission and power electronics, including quasistatic and time-harmonic electromagnetic fields. Additionally, the COMSOL® software features unique capabilities for coupling thermal and structural effects with electromagnetic fields. Effects such as induction heating and Joule heating, as well as structural stresses and strains caused by thermal expansion, can be studied with user-friendly modeling features.

In this webinar, we will provide an overview of the modeling and simulation functionality in the COMSOL® software for the design of high-power cables and devices for power electronics.

Register now for this free webinar!

When Marianne Smith was teaching computer science in 2016 at Flathead Valley Community College, in Kalispell, Mont., the adjunct professor noticed the female students in her class were severely outnumbered, she says.

Smith says she believed the disparity was because girls were not being introduced to science, technology, engineering, and mathematics in elementary and middle school.

Code Girls United

Founded

2018

Headquarters

Kalispell, Mont.

Employees

10

In 2017 she decided to do something to close the gap. The IEEE member started an after-school program to teach coding and computer science.

What began as a class of 28 students held in a local restaurant is now a statewide program run by Code Girls United, a nonprofit Smith founded in 2018. The organization has taught more than 1,000 elementary, middle, and high school students across 38 cities in Montana and three of the state’s Native American reservations. Smith has plans to expand the nonprofit to South Dakota, Wisconsin, and other states, as well as other reservations.

“Computer science is not a K–12 requirement in Montana,” Smith says. “Our program creates this rare hands-on experience that provides students with an experience that’s very empowering for girls in our community.”

The nonprofit was one of seven winners last year of MIT Solve’s Gender Equity in STEM Challenge. The initiative supports organizations that work to address gender barriers. Code Girls United received US $100,000 to use toward its program.

“The MIT Solve Gender Equity in STEM Challenge thoroughly vets all applicants—their theories, practices, organizational health, and impact,” Smith says. “For Code Girls United to be chosen as a winner of the contest is a validating honor.”

From a restaurant basement to statewide programs

When Smith had taught her sons how to program robots, she found that programming introduced a set of logic and communication skills similar to learning a new language, she says.

Those skills were what many girls were missing, she reasoned.

“It’s critical that girls be given the opportunity to speak and write in this coding language,” she says, “so they could also have the chance to communicate their ideas.”

An app to track police vehicles

Last year Code Girls United’s advanced class held in Kalispell received a special request from Jordan Venezio, the city’s police chief. He asked the class to create an app to help the Police Department manage its vehicle fleet.

The department was tracking the location of its police cars on paper, a process that made it challenging to get up-to-date information about which cars were on patrol, available for use, or being repaired, Venezio told the Flathead Beacon.

The objective was to streamline day-to-day vehicle operations. To learn how the department operates and see firsthand the difficulties administrators faced when managing the vehicles, two students shadowed officers for 10 weeks.

The students programmed the app using Visual Studio Code, React Native, Expo Go, and GitHub.

The department’s administrators now more easily can see each vehicle’s availability, whether it’s at the repair shop, or if it has been retired from duty.

“It’s a great privilege for the girls to be able to apply the skills they’ve learned in the Code Girls United program to do something like this for the community,” Smith says. “It really brings our vision full circle.”

At first she wasn’t sure what subjects to teach, she says, reasoning that Java and other programming languages were too advanced for elementary school students.

She came across MIT App Inventor, a block-based visual programming language for creating mobile apps for Android and iOS devices. Instead of learning a coding language by typing it, students drag and drop jigsaw puzzle–like pieces that contain code to issue instructions. She incorporated building an app with general computer science concepts such as conditionals, logic flow, and variables. With each concept learned, the students built a more difficult app.

“It was perfect,” she says, “because the girls could make an app and test it the same day. It’s also very visual.”

Once she had a curriculum, she wanted to find willing students, so she placed an advertisement in the local newspaper. Twenty-eight girls signed up for the weekly classes, which were held in a diner. Assisting Smith were Beth Schecher, a retired technical professional; and Liz Bernau, a newly graduated elementary school teacher who taught technology classes. Students had to supply their own laptop.

At the end of the first 18 weeks, the class was tasked with creating apps to enter in the annual Technovation Girls competition. The contest seeks out apps that address issues including animal abandonment, safely reporting domestic violence, and access to mental health services.

The first group of students created several apps to enter in the competition, including ones that connected users to water-filling stations, provided people with information about food banks, and allowed users to report potholes. The group made it to the competition’s semifinals.

The coding program soon outgrew the diner and moved to a computer lab in a nearby elementary school. From there classes were held at Flathead Valley Community College. The program continued to grow and soon expanded to schools in other Montana towns including Belgrade, Havre, Joliet, and Polson.

The COVID-19 pandemic prompted the program to become virtual—which was “oddly fortuitous,” Smith says. After she made the curriculum available for anyone to use via Google Classroom, it increased in popularity.

That’s when she decided to launch her nonprofit. With that came a new curriculum.

What began as a class of 28 students held in a restaurant in Kalispell, Mont., has grown into a statewide program run by Code Girls United. The nonprofit has taught coding and computer science to more than 1,000 elementary, middle, and high school students. Code Girls United

Program expands across the state

Beginner, intermediate, and advanced classes were introduced. Instructors of the weekly after-school program are volunteers and teachers trained by Smith or one of the organization’s 10 employees. The teachers are paid a stipend.

For the first half of the school year, students in the beginner class learn computer science while creating apps.

“By having them design and build a mobile app,” Smith says, “I and the other teachers teach them computer science concepts in a fun and interactive way.”

Once students master the course, they move on to the intermediate and advanced levels, where they are taught lessons in computer science and learn more complicated programming concepts such as Java and Python.

“It’s important to give girls who live on the reservations educational opportunities to close the gap. It’s the right thing to do for the next generation.”

During the second half of the year, the intermediate and advanced classes participate in Code Girls United’s App Challenge. The girls form teams and choose a problem in their community to tackle. Next they write a business plan that includes devising a marketing strategy, designing a logo, and preparing a presentation. A panel of volunteer judges evaluates their work, and the top six teams receive a scholarship of up to $5,000, which is split among the members.

The organization has given out more than 55 scholarships, Smith says.

“Some of the girls who participated in our first education program are now going to college,” she says. “Seventy-two percent of participants are pursuing a degree in a STEM field, and quite a few are pursuing computer science.”

Introducing coding to Native Americans

The program is taught to high school girls on Montana’s Native American reservations through workshops.

Many reservations lack access to technology resources, Smith says, so presenting the program there has been challenging. But the organization has had some success and is working with the Blackfeet reservation, the Salish and Kootenai tribes on the Flathead reservation, and the Nakota and Gros Ventre tribes at Fort Belknap.

The workshops tailor technology for Native American culture. In the newest course, students program a string of LEDs to respond to the drumbeat of tribal songs using the BBC’s Micro:bit programmable controller. The lights are attached to the bottom of a ribbon skirt, a traditional garment worn by young women. Colorful ribbons are sewn horizontally across the bottom, with each hue having a meaning.

The new course was introduced to students on the Flathead reservation this month.

“Montana’s reservations are some of the most remote and resource-limited communities,” Smith says, “especially in regards to technology and educational opportunities.

“It’s important to give girls who live on the reservations educational opportunities to close the gap. It’s the right thing to do for the next generation.”

One of the biggest challenges for robotics right now is practical autonomous operation in unstructured environments. That is, doing useful stuff in places your robot hasn’t been before and where things may not be as familiar as your robot might like. Robots thrive on predictability, which has put some irksome restrictions on where and how they can be successfully deployed.

But over the past few years, this has started to change, thanks in large part to a couple of pivotal robotics challenges put on by DARPA. The DARPA Subterranean Challenge ran from 2018 to 2021, putting mobile robots through a series of unstructured underground environments. And the currently ongoing DARPA RACER program tasks autonomous vehicles with navigating long distances off-road. Some extremely impressive technology has been developed through these programs, but there’s always a gap between this cutting-edge research and any real-world applications.

Now, a bunch of the folks involved in these challenges, including experienced roboticists from NASA, DARPA, Google DeepMind, Amazon, and Cruise (to name just a few places) are applying everything that they’ve learned to enable real-world practical autonomy for mobile robots at a startup called Field AI.

Field AI was cofounded by Ali Agha, who previously was a group leader for NASA JPL’s Aerial Mobility Group as well as JPL’s Perception Systems Group. While at JPL, Agha led Team CoSTAR, which won the DARPA Subterranean Challenge Urban Circuit. Agha has also been the principal investigator for DARPA RACER, first with JPL, and now continuing with Field AI. “Field AI is not just a startup,” Agha tells us. “It’s a culmination of decades of experience in AI and its deployment in the field.”

Unstructured environments are where things are constantly changing, which can play havoc with robots that rely on static maps.

The “field” part in Field AI is what makes Agha’s startup unique. Robots running Field AI’s software are able to handle unstructured, unmapped environments without reliance on prior models, GPS, or human intervention. Obviously, this kind of capability was (and is) of interest to NASA and JPL, which send robots to places where there are no maps, GPS doesn’t exist, and direct human intervention is impossible.

But DARPA SubT demonstrated that similar environments can be found on Earth, too. For instance, mines, natural caves, and the urban underground are all extremely challenging for robots (and even for humans) to navigate. And those are just the most extreme examples: robots that need to operate inside buildings or out in the wilderness have similar challenges understanding where they are, where they’re going, and how to navigate the environment around them.

An autonomous vehicle drives across kilometers of desert with no prior map, no GPS, and no road.Field AI

Despite the difficulty that robots have operating in the field, this is an enormous opportunity that Field AI hopes to address. Robots have already proven their worth in inspection contexts, typically where you either need to make sure that nothing is going wrong across a large industrial site, or for tracking construction progress inside a partially completed building. There’s a lot of value here because the consequences of something getting messed up are expensive or dangerous or both, but the tasks are repetitive and sometimes risky and generally don’t require all that much human insight or creativity.

Uncharted Territory as Home Base

Where Field AI differs from other robotics companies offering these services, as Agha explains, is that his company wants to do these tasks without first having a map that tells the robot where to go. In other words, there’s no lengthy setup process, and no human supervision, and the robot can adapt to changing and new environments. Really, this is what full autonomy is all about: going anywhere, anytime, without human interaction. “Our customers don’t need to train anything,” Agha says, laying out the company’s vision. “They don’t need to have precise maps. They press a single button, and the robot just discovers every corner of the environment.” This capability is where the DARPA SubT heritage comes in. During the competition, DARPA basically said, “here’s the door into the course. We’re not going to tell you anything about what’s back there or even how big it is. Just go explore the whole thing and bring us back the info we’ve asked for.” Agha’s Team CoSTAR did exactly that during the competition, and Field AI is commercializing this capability.

“With our robots, our aim is for you to just deploy it, with no training time needed. And then we can just leave the robots.” —Ali Agha, Field AI

The other tricky thing about these unstructured environments, especially construction environments, is that things are constantly changing, which can play havoc with robots that rely on static maps. “We’re one of the few, if not the only company that can leave robots for days on continuously changing construction sites with minimal supervision,” Agha tells us. “These sites are very complex—every day there are new items, new challenges, and unexpected events. Construction materials on the ground, scaffolds, forklifts, and heavy machinery moving all over the place, nothing you can predict.”

Field AI

Field AI’s approach to this problem is to emphasize environmental understanding over mapping. Agha says that essentially, Field AI is working towards creating “field foundation models” (FFMs) of the physical world, using sensor data as an input. You can think of FFMs as being similar to the foundation models of language, music, and art that other AI companies have created over the past several years, where ingesting a large amount of data from the Internet enables some level of functionality in a domain without requiring specific training for each new situation. Consequently, Field AI’s robots can understand how to move in the world, rather than just where to move. “We look at AI quite differently from what’s mainstream,” Agha explains. “We do very heavy probabilistic modeling.” Much more technical detail would get into Field AI’s IP, says Agha, but the point is that real-time world modeling becomes a by-product of Field AI’s robots operating in the world rather than a prerequisite for that operation. This makes the robots fast, efficient, and resilient.

Developing field-foundation models that robots can use to reliably go almost anywhere requires a lot of real-world data, which Field AI has been collecting at industrial and construction sites around the world for the past year. To be clear, they’re collecting the data as part of their commercial operations—these are paying customers that Field AI has already. “In these job sites, it can traditionally take weeks to go around a site and map where every single target of interest that you need to inspect is,” explains Agha. “But with our robots, our aim is for you to just deploy it, with no training time needed. And then we can just leave the robots. This level of autonomy really unlocks a lot of use cases that our customers weren’t even considering, because they thought it was years away.” And the use cases aren’t just about construction or inspection or other areas where we’re already seeing autonomous robotic systems, Agha says. “These technologies hold immense potential.”

There’s obviously demand for this level of autonomy, but Agha says that the other piece of the puzzle that will enable Field AI to leverage a trillion dollar market is the fact that they can do what they do with virtually any platform. Fundamentally, Field AI is a software company—they make sensor payloads that integrate with their autonomy software, but even those payloads are adjustable, ranging from something appropriate for an autonomous vehicle to something that a drone can handle.

Heck, if you decide that you need an autonomous humanoid for some weird reason, Field AI can do that too. While the versatility here is important, according to Agha, what’s even more important is that it means you can focus on platforms that are more affordable, and still expect the same level of autonomous performance, within the constraints of each robot’s design, of course. With control over the full software stack, integrating mobility with high-level planning, decision making, and mission execution, Agha says that the potential to take advantage of relatively inexpensive robots is what’s going to make the biggest difference toward Field AI’s commercial success.

Same brain, lots of different robots: the Field AI team’s foundation models can be used on robots big, small, expensive, and somewhat less expensive.Field AI

Field AI is already expanding its capabilities, building on some of its recent experience with DARPA RACER by working on deploying robots to inspect pipelines for tens of kilometers and to transport materials across solar farms. With revenue coming in and a substantial chunk of funding, Field AI has even attracted interest from Bill Gates. Field AI’s participation in RACER is ongoing, under a sort of subsidiary company for federal projects called Offroad Autonomy, and in the meantime its commercial side is targeting expansion to “hundreds” of sites on every platform it can think of, including humanoids.

At TSMC’s North American Technology Symposium on Wednesday, the company detailed both its semiconductor technology and chip-packaging technology road maps. While the former is key to keeping the traditional part of Moore’s Law going, the latter could accelerate a trend toward processors made from more and more silicon, leading quickly to systems the size of a full silicon wafer. Such a system, Tesla’s next generation Dojo training tile is already in production, TSMC says. And in 2027 the foundry plans to offer technology for more complex wafer-scale systems than Tesla’s that could deliver 40 times as much computing power as today’s systems.

For decades chipmakers increased the density of logic on processors largely by scaling down the area that transistors take up and the size of interconnects. But that scheme has been running out of steam for a while now. Instead, the industry is turning to advanced packaging technology that allows a single processor to be made from a larger amount of silicon. The size of a single chip is hemmed in by the largest pattern that lithography equipment can make. Called the reticle limit, that’s currently about 800 square millimeters. So if you want more silicon in your GPU you need to make it from two or more dies. The key is connecting those dies so that signals can go from one to the other as quickly and with as little energy as if they were all one big piece of silicon.

TSMC already makes a wafer-size AI accelerator for Cerebras, but that arrangement appears to be unique and is different from what TSMC is now offering with what it calls System-on-Wafer.

In 2027, you will get a full-wafer integration that delivers 40 times as much compute power, more than 40 reticles’ worth of silicon, and room for more than 60 high-bandwidth memory chips, TSMC predicts

For Cerebras, TSMC makes a wafer full of identical arrays of AI cores that are smaller than the reticle limit. It connects these arrays across the “scribe lines,” the areas between dies that are usually left blank, so the wafer can be diced up into chips. No chipmaking process is perfect, so there are always flawed parts on every wafer. But Cerebras designed in enough redundancy that it doesn’t matter to the finished computer.

However, with its first round of System-on-Wafer, TSMC is offering a different solution to the problems of both reticle limit and yield. It starts with already tested logic dies to minimize defects. (Tesla’s Dojo contains a 5-by-5 grid of pretested processors.) These are placed on a carrier wafer, and the blank spots between the dies are filled in. Then a layer of high-density interconnects is constructed to connect the logic using TSMC’s integrated fan-out technology. The aim is to make data bandwidth among the dies so high that they effectively act like a single large chip.

By 2027, TSMC plans to offer wafer-scale integration based on its more advanced packaging technology, chip-on-wafer-on-substrate (CoWoS). In that technology, pretested logic and, importantly, high-bandwidth memory, is attached to a silicon substrate that’s been patterned with high-density interconnects and shot through with vertical connections called through-silicon vias. The attached logic chips can also take advantage of the company’s 3D-chip technology called system-on-integrated chips (SoIC).

The wafer-scale version of CoWoS is the logical endpoint of an expansion of the packaging technology that’s already visible in top-end GPUs. Nvidia’s next GPU, Blackwell, uses CoWos to integrate more than 3 reticle sizes’ worth of silicon, including 8 high-bandwidth memory (HBM) chips. By 2026, the company plans to expand that to 5.5 reticles, including 12 HBMs. TSMC says that would translate to more than 3.5 times as much compute power as its 2023 tech allows. But in 2027, you can get a full wafer integration that delivers 40 times as much compute, more than 40 reticles’ worth of silicon and room for more than 60 HBMs, TSMC predicts.

What Wafer Scale Is Good For

The 2027 version of system-on-wafer somewhat resembles technology called Silicon-Interconnect Fabric, or Si-IF, developed at UCLA more than five years ago. The team behind SiIF includes electrical and computer-engineering professor Puneet Gupta and IEEE Fellow Subramanian Iyer, who is now charged with implementing the packaging portion of the United States’ CHIPS Act.

Since then, they’ve been working to make the interconnects on the wafer more dense and to add other features to the technology. “If you want this as a full technology infrastructure, it needs to do many other things beyond just providing fine-pitch connectivity,” says Gupta, also an IEEE Fellow. “One of the biggest pain points for these large systems is going to be delivering power.” So the UCLA team is working on ways to add good-quality capacitors and inductors to the silicon substrate and integrating gallium nitride power transistors.

AI training is the obvious first application for wafer-scale technology, but it is not the only one, and it may not even be the best, says University of Illinois Urbana-Champaign computer architect and IEEE Fellow Rakesh Kumar. At the International Symposium on Computer Architecture in June, his team is presenting a design for a wafer-scale network switch for data centers. Such a system could cut the number of advanced network switches in a very large—16,000-rack—data center from 4,608 to just 48, the researchers report. A much smaller, enterprise-scale, data center for say 8,000 servers could get by using a single wafer-scale switch.

Digital Chinese-language keyboards that are vulnerable to spying and eavesdropping have been used by 1 billion smartphone users, according to a new report. The widespread threats these leaky systems reveal could also present a concerning new kind of exploit for cyberattacks, whether the device uses a Chinese-language keyboard, an English keyboard, or any other.

Last year, the University of Toronto’s Citizen Lab released a study of a proprietary Chinese keyboard system owned by the Shenzhen-based tech giant Tencent. Citizen Lab’s “Sogou Keyboard” report exposed the widespread range of attacks possible on the keyboard, which could leak a user’s key presses to outside eavesdroppers. Now, in the group’s new study, released last week, the same researchers have discovered that essentially all the world’s popular Chinese smartphone keyboards have suffered similar vulnerabilities.

“Whatever Chinese-language users of your app might have typed into it has been exposed for years.” —Jedidiah Crandall, Arizona State University

And while the specific bugs the two reports have uncovered have been fixed in most instances, the researchers’ findings—and in particular, their recommendations—point to substantially larger gaps in the systems that extend into software developed around the world, no matter the language.

“All of these keyboards were also using custom network protocols,” says Mona Wang, a computer science Ph.D. student at Princeton University and coauthor of the report. “Because I had studied these sort of custom network protocols before, then this immediately screamed to me that there was something really terrible going on.”

Jedidiah Crandall, an associate professor of computing and augmented intelligence at Arizona State University in Tempe, who was consulted in the report’s preparation but was not on the research team, says these vulnerabilities matter for nearly any coder or development team that releases their work to the world. “If you are a developer of a privacy-focused chat app or an app for tracking something health related, whatever Chinese language users of your app might have typed into it has been exposed for years,” he says.

The Chinese keyboard problem

Chinese, a language of tens of thousands of characters with some 4,000 or more in common use, represents a distinct challenge for keyboard input. A range of different keyboard systems have been developed in the digital era—sometimes called pinyin keyboards, named after a popular romanization system for standard Chinese. Ideally, these creative approaches to digital input enable a profoundly complex language to be straightforwardly phoneticized and transliterated via a compact, often QWERTY-style keyboard format.

“Even competent and well-resourced people get encryption wrong, because it’s really hard to do correctly.” —Mona Wang, Princeton University

Computational and AI smarts can help transform key presses into Chinese characters on the screen. But Chinese keyboards often involve many interchanges across the Internet between cloud servers and other assistive networked apps, just to make it possible for a Chinese-speaking person to be able to type the characters.

According to the report—and an FAQ the researchers released explaining the technical points in plain language—the Chinese keyboards studied all used character-prediction features, which in turn relied on cloud-computing resources. The researchers found that improperly secured communications between a device’s keyboard app and those external cloud servers meant that users’ keystrokes (and therefore their messages) could be accessed in transit.

Jeffrey Knockel, a senior research associate at Citizen Lab and the report coauthor, says cloud-based character prediction is a particularly attractive feature for Chinese-language keyboards, given the vast array of possible characters that any given QWERTY keystroke sequence might be attempting to represent. “If you’re typing in English or any language where there’s enough keys on a keyboard for all your letters, that’s already a much simpler task to design a keyboard around than an ideographic language where you might have over 10,000 characters,” he says.

Chinese-language keyboards are often “pinyin keyboards,” which allow for thousands of characters to be typed using a QWERTY-style approach.Zamoeux/Wikimedia

Sarah Scheffler, a postdoctoral associate at MIT, expressed concern also about other kinds of data vulnerabilities that the Citizen Lab report reveals—beyond keyboards and Chinese-language specific applications necessarily. “The vulnerabilities [identified by the report] are not at all specific to pinyin keyboards,” she says. “It applies to any application sending data over the Internet. Any app sending unencrypted—or badly encrypted—information would have similar issues.”

Wang says the chief problem the researchers uncovered concerns the fact that so many Chinese-keyboard protocols transmit data using inferior and sometimes custom-made encryption.

“These encryption protocols are probably developed by very, very competent and very well-resourced people,” Wang says. “But even competent and well-resourced people get encryption wrong, because it’s really hard to do correctly.”

Beyond the vulnerabilities exposed

Scheffler points to the two-decades-long testing, iteration, and development of the transport layer security (TLS) system underlying much of the Internet’s secure communications, including websites that use the Hypertext Transfer Protocol Secure (HTTPS) protocol. (The first version of TLS was specified and released in 1999.) “All these Chinese Internet companies who are rolling their own [cryptography] or using their own encryption algorithms are sort of missing out on all those 20 years of standard encryption development,” Wang says.

Crandall says the report may have also inadvertently highlighted assumptions about security protocols that may not always apply in every corner of the globe. “Protocols like TLS sometimes make assumptions that don’t suit the needs of developers in certain parts of the world,” he says. For instance, he adds, custom-made, non-TLS security systems may be more attractive “where the network delay is high or where people may spend large amounts of time in areas where the network is not accessible.”

Scheffler says the Chinese-language keyboard problem could even represent a kind of canary in the coal mine for a range of computer, smartphone, and software systems. Because of their reliance on extensive Internet communications, such systems—while perhaps overlooked or relegated to the background by developers—also still represent potential cybersecurity attack surfaces.

“Anecdotally, a lot of these security failures arise from groups that don’t think they’re doing anything that requires security or don’t have much security expertise,” Scheffler says.

Scheffler identifies “Internet-based predictive-text keyboards in any language, and maybe some of the Internet-based AI features that have crept into apps over the years” as possible places concealing cybersecurity vulnerabilities similar to those that the Citizen Lab team discovered in Chinese-language keyboards. This category could include voice recognition, speech-to-text, text-to-speech, and generative AI tools, she adds.

“Security and privacy isn’t many people’s first thought when they’re building their cool image-editing application,” says Scheffler. ”Maybe it shouldn’t be the first thought, but it should definitely be a thought by the time the application makes it to users.”

This story was updated 29 April 2024.

Making breakthroughs in artificial intelligence these days requires huge amounts of computing power. In January, Meta CEO Mark Zuckerberg announced that by the end of this year, the company will have installed 350,000 Nvidia GPUs—the specialized computer chips used to train AI models—to power its AI research.

As a data-center network engineer with Meta’s network infrastructure team, Susana Contrera is playing a leading role in this unprecedented technology rollout. Her job is about “bringing designs to life,” she says. Contrera and her colleagues take high-level plans for the company’s AI infrastructure and turn those blueprints into reality by working out how to wire, power, cool, and house the GPUs in the company’s data centers.

Susana Contrera

Employer:

Meta

Occupation:

Data-center network engineer

Education:

Bachelor’s degree in telecommunications engineering, Andrés Bello Catholic University in Caracas, Venezuela

Contrera, who now works remotely from Florida, has been at Meta since 2013, spending most of that time helping to build the computer systems that support its social media networks, including Facebook and Instagram. But she says that AI infrastructure has become a growing priority, particularly in the past two years, and represents an entirely new challenge. Not only is Meta building some of the world’s first AI supercomputers, it is racing against other companies like Google and OpenAI to be the first to make breakthroughs.

“We are sitting right at the forefront of the technology,” Contrera says. “It’s super challenging, but it’s also super interesting, because you see all these people pushing the boundaries of what we thought we could do.”

Cisco Certification Opened Doors

Growing up in Caracas, Venezuela, Contrera says her first introduction to technology came from playing video games with her older brother. But she decided to pursue a career in engineering because of her parents, who were small-business owners.

“They were always telling me how technology was going to be a game changer in the future, and how a career in engineering could open many doors,” she says.

She enrolled at Andrés Bello Catholic University in Caracas in 2001 to study telecommunications engineering. In her final year, she signed up for the training and certification program to become a Cisco Certified Network Associate. The program covered topics such as the fundamentals of networking and security, IP services, and automation and programmability.

The certificate opened the door to her first job in 2006—managing the computer network of a business-process outsourcing company, Atento, in Caracas.

“Getting your hands dirty can give you a lot of perspective.”

“It was a very large enterprise network that had just the right amount of complexity for a very small team,” she says. “That gave me a lot of freedom to put my knowledge into practice.”

At the time, Venezuela was going through a period of political unrest. Contrera says she didn’t see a future for herself in the country, so she decided to leave for Europe.

She enrolled in a master’s degree program in project management in 2009 at Spain’s Pontifical University of Salamanca, continuing to collect additional certifications through Cisco in her free time. In 2010, partway through the program, she left for a job as a support engineer at the Madrid-based law firm Ecija, which provides legal advice to technology, media, and telecommunications companies. Following that with a stint as a network engineer at Amazon’s facility in Dublin from 2011 to 2013, she then joined Meta and “the rest is history,” she says.

Starting From the Edge Network

Contrera first joined Meta as a network deployment engineer, helping build the company’s “edge” network. In this type of network design, user requests go out to small edge servers dotted around the world instead of to Meta’s main data centers. Edge systems can deal with requests faster and reduce the load on the company’s main computers.

After several years traveling around Europe setting up this infrastructure, she took a managerial position in 2016. But after a couple of years she decided to return to a hands-on role at the company.

“I missed the satisfaction that you get when you’re part of a project, and you can clearly see the impact of solving a complex technical problem,” she says.

Because of the rapid growth of Meta’s services, her work primarily involved scaling up the capacity of its data centers as quickly as possible and boosting the efficiency with which data flowed through the network. But the work she is doing today to build out Meta’s AI infrastructure presents very different challenges, she says.

Designing Data Centers for AI

Training Meta’s largest AI models involves coordinating computation over large numbers of GPUs split into clusters. These clusters are often housed in different facilities, often in distant cities. It’s crucial that messages passing back and forth have very low latency and are lossless—in other words, they move fast and don’t drop any information.

Building data centers that can meet these requirements first involves Meta’s network engineering team deciding what kind of hardware should be used and how it needs to be connected.

“They have to think about how those clusters look from a logical perspective,” Contrera says.

Then Contrera and other members of the network infrastructure team take this plan and figure out how to fit it into Meta’s existing data centers. They consider how much space the hardware needs, how much power and cooling it will require, and how to adapt the communications systems to support the additional data traffic it will generate. Crucially, this AI hardware sits in the same facilities as the rest of Meta’s computing hardware, so the engineers have to make sure it doesn’t take resources away from other important services.

“We help translate these ideas into the real world,” Contrera says. “And we have to make sure they fit not only today, but they also make sense for the long-term plans of how we are scaling our infrastructure.”

Working on a Transformative Technology

Planning for the future is particularly challenging when it comes to AI, Contrera says, because the field is moving so quickly.

“It’s not like there is a road map of how AI is going to look in the next five years,” she says. “So we sometimes have to adapt quickly to changes.”

With today’s heated competition among companies to be the first to make AI advances, there is a lot of pressure to get the AI computing infrastructure up and running. This makes the work much more demanding, she says, but it’s also energizing to see the entire company rallying around this goal.

While she sometimes gets lost in the day-to-day of the job, she loves working on a potentially transformative technology. “It’s pretty exciting to see the possibilities and to know that we are a tiny piece of that big puzzle,” she says.

Hands-on Data Center Experience

For those interested in becoming a network engineer, Contrera says the certification programs run by companies like Cisco are useful. But she says it’s also important not to focus just on simply ticking boxes or rushing through courses just to earn credentials. “Take your time to understand the topics because that’s where the value is,” she says.

It’s good to get some experience working in data centers on infrastructure deployment, she says, because “getting your hands dirty can give you a lot of perspective.” And increasingly, coding can be another useful skill to develop to complement more traditional network engineering capabilities.

Mainly, she says, just “enjoy the ride” because networking can be a truly fascinating topic once you delve in. “There’s this orchestra of protocols and different technologies playing together and interacting,” she says. “I think that’s beautiful.”

I hate the eye strain that often comes with peering through a telescope at the night sky—I’d rather let a camera capture the scene. But I’m too frugal to sink thousands of dollars into high-quality astrophotography gear. The Goldilocks solution for me is something that goes by the name of electronically assisted astronomy, or EAA.

EAA occupies a middle ground in amateur astronomy: more involved than gazing through binoculars or a telescope, but not as complicated as using specialized cameras, expensive telescopes, and motorized tracking mounts. I set about exploring how far I could get doing EAA on a limited budget.

Electronically-assisted-astronomy photographs captured with my rig: the moon [top], the sun [middle], and the Orion Nebula [bottom] David Schneider

First, I purchased a used Canon T6 DSLR on eBay. Because it had a damaged LCD viewscreen and came without a lens, it cost just US $100. Next, rather than trying to marry this camera to a telescope, I decided to get a telephoto lens: Back to eBay for a 40-year-old Nikon 500-mm F/8 “mirror” telephoto lens for $125. This lens combines mirrors and lenses to create a folded optical path. So even though the focal length of this telephoto is a whopping 50 centimeters, the lens itself is only about 15 cm long. A $20 adapter makes it work with the Canon.

The Nikon lens lacks a diaphragm to adjust its aperture and hence its depth of field. Its optical geometry makes things that are out of focus resemble doughnuts. And it can’t be autofocused. But these shortcomings aren’t drawbacks for astrophotography. And the lens has the big advantage that it can be focused beyond infinity. This allows you to adjust the focus on distant objects accurately, even if the lens expands and contracts with changing temperatures.

Getting the focus right is one of the bugaboos of using a telephoto lens for astrophotography, because the focus on such lenses is touchy and easily gets knocked off kilter. To avoid that, I built something (based on a design I found in an online astronomy forum) that clamps to the focus ring and allows precise adjustments using a small knob.

My next purchase was a modified gun sight to make it easier to aim the camera. The version I bought (for $30 on Amazon) included an adapter that let me mount it to my camera’s hot shoe. You’ll also need a tripod, but you can purchase an adequate one for less than $30.

Getting the focus right is one of the bugaboos of using a telephoto lens

The only other hardware you need is a laptop. On my Windows machine, I installed four free programs: Canon’s EOS Utility (which allows me to control the camera and download images directly), Canon’s Digital Photo Professional (for managing the camera’s RAW format image files), the GNU Image Manipulation Program (GIMP) photo editor, and a program called Deep Sky Stacker, which lets me combine short-exposure images to enhance the results without having Earth’s rotation ruin things.

It was time to get started. But focusing on astronomical objects is harder than you might think. The obvious strategy is to put the camera in “live view” mode, aim it at Jupiter or a bright star, and then adjust the focus until the object is as small as possible. But it can still be hard to know when you’ve hit the mark. I got a big assist from what’s known as a Bahtinov mask, a screen with angled slats you temporarily stick in front of the lens to create a diffraction pattern that guides focusing.

Stacking software takes a series of images of the sky, compensates for the motion of the stars, and combines the images to simulate long exposures without blurring.

After getting some good shots of the moon, I turned to another easy target: the sun. That required a solar filter, of course. I purchased one for $9 , which I cut into a circle and glued to a candy tin from which I had cut out the bottom. My tin is of a size that slips perfectly over my lens. With this filter, I was able to take nice images of sunspots. The challenge again was focusing, which required trial and error, because strategies used for stars and planets don’t work for the sun.

With focusing down, the next hurdle was to image a deep-sky object, or DSO—star clusters, galaxies, and nebulae. To image these dim objects really well requires a tracking mount, which turns the camera so that you can take long exposures without blurring from the motion of the Earth. But I wanted to see what I could do without a tracker.

I first needed to figure out how long of an exposure was possible with my fixed camera. A common rule of thumb is to take the focal length of your telescope in millimeters and divide by 500 to give you the maximum exposure duration in seconds. For my setup, that would be 1 second. A more sophisticated approach, called the NPF rule, factors in additional details regarding your imaging sensor. Using an online NPF-rule calculator gave me a slightly lower number: 0.8 seconds. To be even more conservative, I used 0.6-second exposures.

My first DSO target was the Orion Nebula, of which I shot 100 images from my suburban driveway. No doubt, I would have done better from a darker spot. I was mindful, though, to acquire calibration frames—“flats” and “darks” and “bias images”—which are used to compensate for imperfections in the imaging system. Darks and bias images are easy enough to obtain by leaving the lens cap on. Taking flats, however, requires an even, diffuse light source. For that I used a $17 A5-size LED tracing pad placed on a white T-shirt covering the lens.

With all these images in hand, I fired up the Deep Sky Stacker program and put it to work. The resultant stack didn’t look promising, but postprocessing in GIMP turned it into a surprisingly detailed rendering of the Orion Nebula. It doesn’t compare, of course, with what somebody can do with a better gear. But it does show the kinds of fascinating images you can generate with some free software, an ordinary DSLR, and a vintage telephoto lens pointed at the right spot.

This article appears in the May 2024 print issue as “Electronically Assisted Astronomy.”

The blistering late-afternoon wind ripped across Camp Taji, a sprawling U.S. military base just north of Baghdad. In a desolate corner of the outpost, where the feared Iraqi Republican Guard had once manufactured mustard gas, nerve agents, and other chemical weapons, a group of American soldiers and Marines were solemnly gathered around an open grave, dripping sweat in the 114-degree heat. They were paying their final respects to Boomer, a fallen comrade who had been an indispensable part of their team for years. Just days earlier, he had been blown apart by a roadside bomb.

As a bugle mournfully sounded the last few notes of “Taps,” a soldier raised his rifle and fired a long series of volleys—a 21-gun salute. The troops, which included members of an elite army unit specializing in explosive ordnance disposal (EOD), had decorated Boomer posthumously with a Bronze Star and a Purple Heart. With the help of human operators, the diminutive remote-controlled robot had protected American military personnel from harm by finding and disarming hidden explosives.

Boomer was a Multi-function Agile Remote-Controlled robot, or MARCbot, manufactured by a Silicon Valley company called Exponent. Weighing in at just over 30 pounds, MARCbots look like a cross between a Hollywood camera dolly and an oversized Tonka truck. Despite their toylike appearance, the devices often leave a lasting impression on those who work with them. In an online discussion about EOD support robots, one soldier wrote, “Those little bastards can develop a personality, and they save so many lives.” An infantryman responded by admitting, “We liked those EOD robots. I can’t blame you for giving your guy a proper burial, he helped keep a lot of people safe and did a job that most people wouldn’t want to do.”

A Navy unit used a remote-controlled vehicle with a mounted video camera in 2009 to investigate suspicious areas in southern Afghanistan.Mass Communication Specialist 2nd Class Patrick W. Mullen III/U.S. Navy

But while some EOD teams established warm emotional bonds with their robots, others loathed the machines, especially when they malfunctioned. Take, for example, this case described by a Marine who served in Iraq:

My team once had a robot that was obnoxious. It would frequently accelerate for no reason, steer whichever way it wanted, stop, etc. This often resulted in this stupid thing driving itself into a ditch right next to a suspected IED. So of course then we had to call EOD [personnel] out and waste their time and ours all because of this stupid little robot. Every time it beached itself next to a bomb, which was at least two or three times a week, we had to do this. Then one day we saw yet another IED. We drove him straight over the pressure plate, and blew the stupid little sh*thead of a robot to pieces. All in all a good day.

Some battle-hardened warriors treat remote-controlled devices like brave, loyal, intelligent pets, while others describe them as clumsy, stubborn clods. Either way, observers have interpreted these accounts as unsettling glimpses of a future in which men and women ascribe personalities to artificially intelligent war machines.

Some battle-hardened warriors treat remote-controlled devices like brave, loyal, intelligent pets, while others describe them as clumsy, stubborn clods.

From this perspective, what makes robot funerals unnerving is the idea of an emotional slippery slope. If soldiers are bonding with clunky pieces of remote-controlled hardware, what are the prospects of humans forming emotional attachments with machines once they’re more autonomous in nature, nuanced in behavior, and anthropoid in form? And a more troubling question arises: On the battlefield, will Homo sapiens be capable of dehumanizing members of its own species (as it has for centuries), even as it simultaneously humanizes the robots sent to kill them?

As I’ll explain, the Pentagon has a vision of a warfighting force in which humans and robots work together in tight collaborative units. But to achieve that vision, it has called in reinforcements: “trust engineers” who are diligently helping the Department of Defense (DOD) find ways of rewiring human attitudes toward machines. You could say that they want more soldiers to play “Taps” for their robot helpers and fewer to delight in blowing them up.

The Pentagon’s Push for Robotics

For the better part of a decade, several influential Pentagon officials have relentlessly promoted robotic technologies, promising a future in which “humans will form integrated teams with nearly fully autonomous unmanned systems, capable of carrying out operations in contested environments.”

Soldiers test a vertical take-off-and-landing drone at Fort Campbell, Ky., in 2020.U.S. Army

As The New York Times reported in 2016: “Almost unnoticed outside defense circles, the Pentagon has put artificial intelligence at the center of its strategy to maintain the United States’ position as the world’s dominant military power.” The U.S. government is spending staggering sums to advance these technologies: For fiscal year 2019, the U.S. Congress was projected to provide the DOD with US $9.6 billion to fund uncrewed and robotic systems—significantly more than the annual budget of the entire National Science Foundation.

Arguments supporting the expansion of autonomous systems are consistent and predictable: The machines will keep our troops safe because they can perform dull, dirty, dangerous tasks; they will result in fewer civilian casualties, since robots will be able to identify enemies with greater precision than humans can; they will be cost-effective and efficient, allowing more to get done with less; and the devices will allow us to stay ahead of China, which, according to some experts, will soon surpass America’s technological capabilities.

Former U.S. deputy defense secretary Robert O. Work has argued for more automation within the military. Center for a New American Security

Among the most outspoken advocate of a roboticized military is Robert O. Work, who was nominated by President Barack Obama in 2014 to serve as deputy defense secretary. Speaking at a 2015 defense forum, Work—a barrel-chested retired Marine Corps colonel with the slight hint of a drawl—described a future in which “human-machine collaboration” would win wars using big-data analytics. He used the example of Lockheed Martin’s newest stealth fighter to illustrate his point: “The F-35 is not a fighter plane, it is a flying sensor computer that sucks in an enormous amount of data, correlates it, analyzes it, and displays it to the pilot on his helmet.”

The beginning of Work’s speech was measured and technical, but by the end it was full of swagger. To drive home his point, he described a ground combat scenario. “I’m telling you right now,” Work told the rapt audience, “10 years from now if the first person through a breach isn’t a friggin’ robot, shame on us.”

“The debate within the military is no longer about whether to build autonomous weapons but how much independence to give them,” said a 2016 New York Times article. The rhetoric surrounding robotic and autonomous weapon systems is remarkably similar to that of Silicon Valley, where charismatic CEOs, technology gurus, and sycophantic pundits have relentlessly hyped artificial intelligence.

For example, in 2016, the Defense Science Board—a group of appointed civilian scientists tasked with giving advice to the DOD on technical matters—released a report titled “Summer Study on Autonomy.” Significantly, the report wasn’t written to weigh the pros and cons of autonomous battlefield technologies; instead, the group assumed that such systems will inevitably be deployed. Among other things, the report included “focused recommendations to improve the future adoption and use of autonomous systems [and] example projects intended to demonstrate the range of benefits of autonomy for the warfighter.”

What Exactly Is a Robot Soldier?

The author’s book, War Virtually, is a critical look at how the U.S. military is weaponizing technology and data.University of California Press

Early in the 20th century, military and intelligence agencies began developing robotic systems, which were mostly devices remotely operated by human controllers. But microchips, portable computers, the Internet, smartphones, and other developments have supercharged the pace of innovation. So, too, has the ready availability of colossal amounts of data from electronic sources and sensors of all kinds. The Financial Times reports: “The advance of artificial intelligence brings with it the prospect of robot-soldiers battling alongside humans—and one day eclipsing them altogether.” These transformations aren’t inevitable, but they may become a self-fulfilling prophecy.

All of this raises the question: What exactly is a “robot-soldier”? Is it a remote-controlled, armor-clad box on wheels, entirely reliant on explicit, continuous human commands for direction? Is it a device that can be activated and left to operate semiautonomously, with a limited degree of human oversight or intervention? Is it a droid capable of selecting targets (using facial-recognition software or other forms of artificial intelligence) and initiating attacks without human involvement? There are hundreds, if not thousands, of possible technological configurations lying between remote control and full autonomy—and these differences affect ideas about who bears responsibility for a robot’s actions.

The U.S. military’s experimental and actual robotic and autonomous systems include a vast array of artifacts that rely on either remote control or artificial intelligence: aerial drones; ground vehicles of all kinds; sleek warships and submarines; automated missiles; and robots of various shapes and sizes—bipedal androids, quadrupedal gadgets that trot like dogs or mules, insectile swarming machines, and streamlined aquatic devices resembling fish, mollusks, or crustaceans, to name a few.

Members of a U.S. Air Force squadron test out an agile and rugged quadruped robot from Ghost Robotics in 2023.Airman First Class Isaiah Pedrazzini/U.S. Air Force

The transitions projected by military planners suggest that servicemen and servicewomen are in the midst of a three-phase evolutionary process, which begins with remote-controlled robots, in which humans are “in the loop,” then proceeds to semiautonomous and supervised autonomous systems, in which humans are “on the loop,” and then concludes with the adoption of fully autonomous systems, in which humans are “out of the loop.” At the moment, much of the debate in military circles has to do with the degree to which automated systems should allow—or require—human intervention.

“Ten years from now if the first person through a breach isn’t a friggin’ robot, shame on us.” —Robert O. Work

In recent years, much of the hype has centered around that second stage: semiautonomous and supervised autonomous systems that DOD officials refer to as “human-machine teaming.” This idea suddenly appeared in Pentagon publications and official statements after the summer of 2015. The timing probably wasn’t accidental; it came at a time when global news outlets were focusing attention on a public backlash against lethal autonomous weapon systems. The Campaign to Stop Killer Robots was launched in April 2013 as a coalition of nonprofit and civil society organizations, including the International Committee for Robot Arms Control, Amnesty International, and Human Rights Watch. In July 2015, the campaign released an open letter warning of a robotic arms race and calling for a ban on the technologies. Cosigners included world-renowned physicist Stephen Hawking, Tesla founder Elon Musk, Apple cofounder Steve Wozniak, and thousands more.

In November 2015, Work gave a high-profile speech on the importance of human-machine teaming, perhaps hoping to defuse the growing criticism of “killer robots.” According to one account, Work’s vision was one in which “computers will fly the missiles, aim the lasers, jam the signals, read the sensors, and pull all the data together over a network, putting it into an intuitive interface humans can read, understand, and use to command the mission”—but humans would still be in the mix, “using the machine to make the human make better decisions.” From this point forward, the military branches accelerated their drive toward human-machine teaming.

The Doubt in the Machine

But there was a problem. Military experts loved the idea, touting it as a win-win: Paul Scharre, in his book Army of None: Autonomous Weapons and the Future of War, claimed that “we don’t need to give up the benefits of human judgment to get the advantages of automation, we can have our cake and eat it too.” However, personnel on the ground expressed—and continue to express—deep misgivings about the side effects of the Pentagon’s newest war machines.

The difficulty, it seems, is humans’ lack of trust. The engineering challenges of creating robotic weapon systems are relatively straightforward, but the social and psychological challenges of convincing humans to place their faith in the machines are bewilderingly complex. In high-stakes, high-pressure situations like military combat, human confidence in autonomous systems can quickly vanish. The Pentagon’s Defense Systems Information Analysis Center Journal noted that although the prospects for combined human-machine teams are promising, humans will need assurances:

[T]he battlefield is fluid, dynamic, and dangerous. As a result, warfighter demands become exceedingly complex, especially since the potential costs of failure are unacceptable. The prospect of lethal autonomy adds even greater complexity to the problem [in that] warfighters will have no prior experience with similar systems. Developers will be forced to build trust almost from scratch.

In a 2015 article, U.S. Navy Commander Greg Smith provided a candid assessment of aviators’ distrust in aerial drones. After describing how drones are often intentionally separated from crewed aircraft, Smith noted that operators sometimes lose communication with their drones and may inadvertently bring them perilously close to crewed airplanes, which “raises the hair on the back of an aviator’s neck.” He concluded:

[I]n 2010, one task force commander grounded his manned aircraft at a remote operating location until he was assured that the local control tower and UAV [unmanned aerial vehicle] operators located halfway around the world would improve procedural compliance. Anecdotes like these abound…. After nearly a decade of sharing the skies with UAVs, most naval aviators no longer believe that UAVs are trying to kill them, but one should not confuse this sentiment with trusting the platform, technology, or [drone] operators.

U.S. Marines [top] prepare to launch and operate a MQ-9A Reaper drone in 2021. The Reaper [bottom] is designed for both high-altitude surveillance and destroying targets.Top: Lance Cpl. Gabrielle Sanders/U.S. Marine Corps; Bottom: 1st Lt. John Coppola/U.S. Marine Corps

Yet Pentagon leaders place an almost superstitious trust in those systems, and seem firmly convinced that a lack of human confidence in autonomous systems can be overcome with engineered solutions. In a commentary, Courtney Soboleski, a data scientist employed by the military contractor Booz Allen Hamilton, makes the case for mobilizing social science as a tool for overcoming soldiers’ lack of trust in robotic systems.

The problem with adding a machine into military teaming arrangements is not doctrinal or numeric…it is psychological. It is rethinking the instinctual threshold required for trust to exist between the soldier and machine.… The real hurdle lies in surpassing the individual psychological and sociological barriers to assumption of risk presented by algorithmic warfare. To do so requires a rewiring of military culture across several mental and emotional domains.… AI [artificial intelligence] trainers should partner with traditional military subject matter experts to develop the psychological feelings of safety not inherently tangible in new technology. Through this exchange, soldiers will develop the same instinctual trust natural to the human-human war-fighting paradigm with machines.

The Military’s Trust Engineers Go to Work

Soon, the wary warfighter will likely be subjected to new forms of training that focus on building trust between robots and humans. Already, robots are being programmed to communicate in more human ways with their users for the explicit purpose of increasing trust. And projects are currently underway to help military robots report their deficiencies to humans in given situations, and to alter their functionality according to the machine’s perceived emotional state of the user.

At the DEVCOM Army Research Laboratory, military psychologists have spent more than a decade on human experiments related to trust in machines. Among the most prolific is Jessie Chen, who joined the lab in 2003. Chen lives and breathes robotics—specifically “agent teaming” research, a field that examines how robots can be integrated into groups with humans. Her experiments test how humans’ lack of trust in robotic and autonomous systems can be overcome—or at least minimized.

For example, in one set of tests, Chen and her colleagues deployed a small ground robot called an Autonomous Squad Member that interacted and communicated with infantrymen. The researchers varied “situation-awareness-based agent transparency”—that is, the robot’s self-reported information about its plans, motivations, and predicted outcomes—and found that human trust in the robot increased when the autonomous “agent” was more transparent or honest about its intentions.

The Army isn’t the only branch of the armed services researching human trust in robots. The U.S. Air Force Research Laboratory recently had an entire group dedicated to the subject: the Human Trust and Interaction Branch, part of the lab’s 711th Human Performance Wing, located at Wright-Patterson Air Force Base, in Ohio.

In 2015, the Air Force began soliciting proposals for “research on how to harness the socio-emotional elements of interpersonal team/trust dynamics and inject them into human-robot teams.” Mark Draper, a principal engineering research psychologist at the Air Force lab, is optimistic about the prospects of human-machine teaming: “As autonomy becomes more trusted, as it becomes more capable, then the Airmen can start off-loading more decision-making capability on the autonomy, and autonomy can exercise increasingly important levels of decision-making.”

Air Force researchers are attempting to dissect the determinants of human trust. In one project, they examined the relationship between a person’s personality profile (measured using the so-called Big Five personality traits: openness, conscientiousness, extraversion, agreeableness, neuroticism) and his or her tendency to trust. In another experiment, entitled “Trusting Robocop: Gender-Based Effects on Trust of an Autonomous Robot,” Air Force scientists compared male and female research subjects’ levels of trust by showing them a video depicting a guard robot. The robot was armed with a Taser, interacted with people, and eventually used the Taser on one. Researchers designed the scenario to create uncertainty about whether the robot or the humans were to blame. By surveying research subjects, the scientists found that women reported higher levels of trust in “Robocop” than men.

The issue of trust in autonomous systems has even led the Air Force’s chief scientist to suggest ideas for increasing human confidence in the machines, ranging from better android manners to robots that look more like people, under the principle that

good HFE [human factors engineering] design should help support ease of interaction between humans and AS [autonomous systems]. For example, better “etiquette” often equates to better performance, causing a more seamless interaction. This occurs, for example, when an AS avoids interrupting its human teammate during a high workload situation or cues the human that it is about to interrupt—activities that, surprisingly, can improve performance independent of the actual reliability of the system. To an extent, anthropomorphism can also improve human-AS interaction, since people often trust agents endowed with more humanlike features…[but] anthropomorphism can also induce overtrust.

It’s impossible to know the degree to which the trust engineers will succeed in achieving their objectives. For decades, military trainers have trained and prepared newly enlisted men and women to kill other people. If specialists have developed simple psychological techniques to overcome the soldier’s deeply ingrained aversion to destroying human life, is it possible that someday, the warfighter might also be persuaded to unquestioningly place his or her trust in robots?

Video Friday is your weekly selection of awesome robotics videos, collected by your friends at IEEE Spectrum robotics. We also post a weekly calendar of upcoming robotics events for the next few months. Please send us your events for inclusion.

Eurobot Open 2024: 8–11 May 2024, LA ROCHE-SUR-YON, FRANCE

ICRA 2024: 13–17 May 2024, YOKOHAMA, JAPAN

RoboCup 2024: 17–22 July 2024, EINDHOVEN, NETHERLANDS

Cybathlon 2024: 25–27 October 2024, ZURICH

Enjoy today’s videos!

DARPA’s Robotic Autonomy in Complex Environments with Resiliency (RACER) program recently conducted its fourth experiment (E4) to assess the performance of off-road unmanned vehicles. These tests, conducted in Texas in late 2023, were the first time the program tested its new vehicle, the RACER Heavy Platform (RHP). The video shows autonomous route following for mobility testing and demonstration, including sensor point cloud visualizations.

The 12-ton RHP is significantly larger than the 2-ton RACER Fleet Vehicles (RFVs) already in use in the program. Using the algorithms on a very different platform helps RACER toward its goal of platform agnostic autonomy of combat-scale vehicles in complex, mission-relevant off-road environments that are significantly more unpredictable than on-road conditions.

[ DARPA ]

In our new Science Robotics paper, we introduce an autonomous navigation system developed for our wheeled-legged quadrupeds, designed for fast and efficient navigation within large urban environments. Driven by neural network policies, our simple, unified control system enables smooth gait transitions, smart navigation planning, and highly responsive obstacle avoidance in populated urban environments.

[ Github ]

Generation 7 of “Phoenix” robots include improved human-like range of motion. Improvements in uptime, visual perception, and tactile sensing increase the capability of the system to perform complex tasks over longer periods. Design iteration significantly decreases build time. The speed at which new tasks can be automated has increased 50x, marking a major inflection point in task automation speed.

[ Sanctuary AI ]

We’re proud to celebrate our one millionth commercial delivery—that’s a million deliveries of lifesaving blood, critical vaccines, last-minute groceries, and so much more. But the best part? This is just the beginning.

[ Zipline ]

Work those hips!

[ RoMeLa ]

This thing is kind of terrifying, and I’m fascinated by it.

[ AVFL ]

We propose a novel humanoid TWIMP, which combines a human mimetic musculoskeletal upper limb with a two-wheel inverted pendulum. By combining the benefit of a musculoskeletal humanoid, which can achieve soft contact with the external environment, and the benefit of a two-wheel inverted pendulum with a small footprint and high mobility, we can easily investigate learning control systems in environments with contact and sudden impact.

From Humanoids 2018.

[ Paper ] via [ JSK Lab ]

Thanks, Kento!

Ballbots are uniquely capable of pushing wheelchairs—arguably better than legged platforms, because they can move in any direction without having to reposition themselves.

[ Paper ]

Charge Robotics is building robots that automate the most labor-intensive parts of solar construction. Solar has rapidly become the cheapest form of power generation in many regions. Demand has skyrocketed, and now the primary barrier to getting it installed is labor logistics and bandwidth. Our robots remove the labor bottleneck, allowing construction companies to meet the rising demand for solar, and enabling the world to switch to renewables faster.

[ Charge Robotics ]

Robots doing precision assembly is cool and all, but those vibratory bowl sorters seem like magic.

[ FANUC ]

The QUT CGRAS project’s robot prototype captures images of baby corals, destined for the Great Barrier Reef, monitoring and counting them in grow tanks. The team uses state-of-the-art AI algorithms to automatically detect and count these coral babies and track their growth over time – saving human counting time and money.

[ QUT ]

We are conducting research to develop Unmanned Aerial Systems to aid in wildfire monitoring. The hazardous, dynamic, and visually degraded environment of wildfire gives rise to many unsolved fundamental research challenges.

[ CMU ]

Here’s a little more video of that robot elevator, but I’m wondering why it’s so slow—clamp those bots in there and rocket that elevator up and down!

[ NAVER ]

In March 2024, Northwestern University’s Center for Robotics and Biosystems demonstrated the Omnid mobile collaborative robots (mocobots) at MARS, a conference in Ojai, California on Machine learning, Automation, Robotics, and Space, hosted by Jeff Bezos. The “swarm” of mocobots is designed to collaborate with humans, allowing a human to easily manipulate large, heavy, or awkward payloads. In this case, the mocobots cancel the effect of gravity, so the human can easily manipulate the mock airplane wing in six degrees of freedom. In general, human-cobot systems combine the best of human capabilities with the best of robot capabilities.

[ Northwestern ]

There’s something so soothing about watching a lithium battery get wrecked and burn for 8 minutes.

[ Hardcore Robotics ]

EELS, or Exobiology Extant Life Surveyor, is a versatile, snake-like robot designed for exploration of previously inaccessible terrain. This talk on EELS was presented at the 2024 Amazon MARS conference.

[ JPL ]

The convergence of AI and robotics will unlock a wonderful new world of possibilities in everyday life, says robotics and AI pioneer Daniela Rus. Diving into the way machines think, she reveals how “liquid networks”—a revolutionary class of AI that mimics the neural processes of simple organisms—could help intelligent machines process information more efficiently and give rise to “physical intelligence” that will enable AI to operate beyond digital confines and engage dynamically in the real world.

[ TED ]

“How did you find me?” specialty coffee roaster Dajo Aertssen asked. He’d just handed me a bag of single-origin cascara, the dried flesh of coffee cherries, in his shop, Cafés Muda in Lille, France.

“The AI sent us,” I replied.

He looked puzzled, so I explained that my companion Dawn and I had asked an app called Perplexity AI to name a “coffee roaster in central Lille,” and that it suggested four places, including Muda. It had linked to a Web page that mentioned Aertssen’s French and world cup tasting championships and that he sources the kinds of beans I enjoy, like Muda’s sidra from Colombia, which tastes like raspberry jam. When I alluded to his status as a supertaster, Aertssen beamed, astonished that I knew anything about him at all.

Dawn and I used Perplexity as an on-demand travel guide throughout our tour of the Flemish region of Belgium and Northern France. The chatbot-based, generative-search startup was founded in 2022 and backed by Nvidia, Jeff Bezos, and New Enterprise Associates, among others. The company’s app rests on GPT-3.5 (the free version) or GPT-4 (the paid version, which I used) and incorporates the company’s own page-ranking algorithm, large language model, and natural-language-processing system.

Perplexity aims to provide an alternative to the broken Google search experience, which forces users to sift through ads, paid placements, and AI-generated junk posts to find what they’re looking for. Perplexity gives answers to questions along with links to the sources it uses. According to search engine optimization company BrightEdge, website referral traffic from Perplexity has been increasing 40 percent per month since January.

As we criss-crossed the countryside seeking cycling museums, waffles, and beer, we passed dozens of wind turbines, prompting me to ask “How much of France’s total energy generation comes from wind power?” (Perplexity didn’t exactly answer this one, but referenced an International Energy Agency site that stated wind met 7.8 percent of national electricity demand in 2021.) Next: “Where can I buy distilled water near me right now?” (In the nearest Carrefour hypermarket, in the housekeeping section, it turns out.) Then we flipped on the audio response mode so the vaguely English-accented male voice could “tell us about the construction of the cathedral in Amiens, France,” which we were going to the next day.

Despite the occasional miss, we agreed that the app significantly enhanced our real-world experience and connected us to Flemish culture and people.

Those connections would not have been as convenient, or maybe even possible, without the Internet—which provides the data on which generative search and many popular large language models rely—and without the seminal contributions of this year’s Medal of Honor recipient, Bob Kahn, whom Senior Editor Tekla S. Perry profiles on page 36 in this issue.

Kahn is being honored for his work on packet communication technologies, which were part of the project that became the ARPANET and the foundations of the Internet. Could he have foreseen people driving around asking an AI-powered search app any question that popped into their heads? Probably not. As Perry writes: “Other than generally thinking that it would be nice if computers could talk to one another, Kahn didn’t give much thought to applications.”

“If you were engineering the Bell system,” Kahn told Perry, “you weren’t trying to figure out who in San Francisco is going to say what to whom in New York. You were just trying to figure out how to enable conversations.”

Kahn’s creations are enabling conversations today, with chatbots in cars and coffee roasters in cafés around the world.

This article appears in the May 2024 print issue as “Making Connections.”

What’s a secret to getting more students to participate in an IEEE society? Give them a seat at the table so they have a say in how the organization is run.

That’s what the IEEE Robotics and Automation Society has done. Budding engineers serve on the RAS board of directors, have voting privileges, and work within technical committees.

“They have been given a voice in how the society runs because, in the end, students are among the main beneficiaries,” says Enrica Tricomi, chair of the RAS’s student activities committee. The SAC is responsible for student programs and benefits. It also makes recommendations to the society’s board about new offerings.

A Guide for Inspiring the Next Generation Roboticists

The IEEE Robotics and Automation Society isn’t focused only on boosting its student membership. It also wants to get more young people interested in pursuing a robotics career. One way the society’s volunteers try to inspire the next generation of roboticists is through IEEE Spectrum’s award-winning Robots website. The interactive guide features more than 250 real-world robots, with thousands of photos, videos, and exclusive interactives, plus news and detailed technical specifications.

The site is designed for anyone interested in robotics, including expert and beginner enthusiasts, researchers, entrepreneurs, students, STEM educators, and other teachers.

Schools and students across the globe use the site. Volunteers on the RAS steering committee suggest robots to add, and they help support new content creation on the site.

“You feel listened to and valued whenever there are official decisions to be made, because the board also wants to know the perspective of students on how to offer benefits to the RAS members, especially for young researchers, since hopefully they will be the society’s future leaders,” says Tricomi, a bioengineer who is pursuing a Ph.D. in robotics at Heidelberg University, in Germany.

The society’s approach has paid off. Since 2018, student membership has grown by more than 50 percent to 5,436. The number of society chapters at student branches has increased from 312 in 2021 to 450.

The ability to express opinions isn’t the only reason students are joining, Tricomi says. The society recently launched several programs to engage them, including career fairs, travel grants, and networking opportunities with researchers.

Giving students leadership opportunities

As SAC chair, Tricomi is a voting member of RAS’s administrative committee, which oversees the society’s operations. She says having voting privileges shows “how important it is to the society to have student representation.”

“We receive a lot of support from the highest levels of the society, specifically the society president, Aude Billard, and past president Frank Chongwoo Park,” Tricomi says. “RAS boards have been rejuvenated to engage students even more and represent their voices. The chairs of these boards—including technical activities, conference activities, and publication activities—want to know the SAC chair and cochairs’ opinion on whether the new activities are benefiting students.”

Student members now can serve on IEEE technical committees that involve robotics in the role of student representatives.

That was an initiative from Kyujin Cho, IEEE Technical Activities vice president. Tricomi says the designation benefits young engineers because they learn about ongoing research in their field and because they have direct access to researchers.

Student representatives also help organize conference workshops.

The students had a hand in creating a welcome kit for conference attendees. The initiative, led by Amy Kyungwon Han, Technical Activities associate vice president, lists each day’s activities and their location.

“I think that all of us, especially those who are younger, can actively contribute and make a difference not only for the society and for ourselves but also for our peers.”

Being engaged with the technical topic in which the students work provides them with career growth, visibility in their field, and an opportunity to share their point of view with peers, Tricomi says.

“Being young, the first time that you express your opinion in public, you always feel uncomfortable because you don’t have much experience,” she says. “This is the opposite of the message the society wants to send. We want to listen to students’ voices because they are an important part of the society.”

Tricomi herself recently became a member of the Technical Activities board.

She joined, she says, because “this is kind of a technical family by choice. And you want to be active and contribute to your family, right? I think that all of us, especially those who are younger, can actively contribute and make a difference not only for the society and for ourselves but also for our peers.”

Job fairs and travel grants

Several new initiatives have been rolled out at the society’s flagship conferences. The meetings have always included onsite events for students to network with each other and to mingle with researchers over lunch. The events give the budding engineers an opportunity to talk with leaders they normally wouldn’t meet, Tricomi says.

“It’s much appreciated, especially by very young or shy students,” she says.

Some luncheons have included sessions on career advice from leaders in academia and industry, or from startup founders—giving the students a sense of what it’s like to work for such organizations.

Conferences now include career fairs, where students can meet with hiring companies.

The society also developed a software platform that allows candidates to upload their résumé onsite. If they are a match for an open position, interviews can be held on the spot.

A variety of travel grants have been made available to students with limited resources so they can present their research papers at the society’s major conferences. More than 200 travel grants were awarded to the 2023 IEEE International Conference on Robotics and Automation, Tricomi says.

“It’s very important for them to be there, presenting their work, gaining visibility, sharing their research, and also networking,” she says.

The new IDEA (inclusion, diversity, equity, and accessibility) travel grant for underrepresented groups was established by the society’s IEEE Women in Engineering committee and its chair, Karinne Ramirez Amaro. The grant can help students who are not presenters to attend conferences. It also helps increase diversity within the robotics field, Tricomi says.

The Member Support Program is a new initiative from the RAS member activities board’s vice president, Katja Mombaur, and past vice president Stefano Stramigioli. Financial support to attend the annual International Conference on Intelligent Robots and Systems is available to members and students who have contributed to the society’s mission-related activities. The projects include organizing workshops, discussions, lectures, or networking events at conferences or sponsored events; serving on boards or committees; or writing papers that were accepted for publication by conferences or journals.

The society also gets budding engineers involved in publication activities through its Young Reviewers Program, which introduces them to best practices for peer review. Senior reviewers assign the students papers to check and oversee their work.

Personal and professional growth opportunities

Tricomi joined the society in 2021 shortly after starting her Ph.D. program at Heidelberg. Her research is in wearable assistive robotics for human augmentation or rehabilitation purposes. She holds a master’s degree in biomedical engineering from Politecnico di Torino, in Italy.

She was new to the field of robotics, so her Ph.D. advisor, IEEE Senior Member Lorenzo Masia, encouraged her to volunteer for the society. She is now transitioning to the role of SAC senior chair, and she says she is eager to collaborate with the new team to promote student and early career engagement within the robotics field.

“I’ve realized I’ve grown up a lot in the two years since I started as chair,” she says. “At the beginning, I was much shier. I really want my colleagues to experience the same personal and professional growth as I have. You learn not only technical skills but also soft skills, which are very important in your career.”

Ever since Return of the Jedi premiered in 1983, people have been imagining the day when they, like the film’s protagonist Luke Skywalker, would get to ride speeder bikes that zip across the landscape while hovering just a few meters above the ground. In the intervening years, there have been numerous claims made by companies that they’ve figured out how to make a real-world product that mimics movie magic. Their demos and PR campaigns have continued to whet the public’s appetite for hoverbikes, but there are some solid reasons why the nascent hoverbike industry has yet to get airborne.

“it’s gonna happen, but I don’t know if it’ll happen in your lifetime and mine,” says Ronald Barrett-Gonzalez, an aerospace-engineering professor at the University of Kansas. “With the current approaches, I think it’s just going to be a while.”

Barrett-Gonzalez was the advisor for a group of University of Kansas aerospace-engineering grad students who participated in the GoFly competition, sponsored by Boeing. The challenge—in which 3,800 teams from 100 countries participated—was to “design and build a safe, quiet, ultra-compact, near-VTOL personal flying device capable of flying 20 miles [32 kilometers] while carrying a single person.”

The eventual grand prize winner was to have been awarded US $1 million, and $250,000 prizes were supposed to have gone to the quietest and the smallest compliant GoFly entries when the challenge concluded in September 2023. But the scheduled final fly-off between the teams whose personal aircraft were selected as the best-built entries in the second of the competition’s three phases was canceled because windy conditions made it unsafe for the machines to take to the skies.

Solving the Physics of Hoverbikes

“Helicopters, for a long time, have been built with relatively small engines for their size,” says Barrett-Gonzalez. “The way that such large vehicles can be lifted by small engines is that they have large rotor diameters. The volume of the column of air that they stand on is great. If you look at hoverbikes, the diameters of their rotors are much smaller. And physics says that you need more power per unit weight to lift an aircraft like that.”

To get the efficiency that comes along with a helicopter’s extensive rotor sweep, hoverbike designers will likely have to give up the thought of these machines touching down in parking spots meant for cars, or at least wait for new generations of engines and electric motors with greater power density to appear along with batteries capable of delivering a lot more power and storing a lot more energy than those available today.

Assessing Hoverbikes’ Risks

Safety concerns are just as big a hurdle to making hoverbikes available for sale. The University of Kansas team’s GoFly entry, called Mamba, had been one of 10 Phase I winners for best design. The six-rotored hexcopter, which emphasized safety, certifiability, and performance, featured shrouded rotors and a tilting stabilizer surface.

The University of Kansas’ GoFly entry is a red-and-black hexcopter with two large and two small horizontal rotors, and two vertically placed rotors in the back. University of Kansas

But Mamba didn’t make it through Phase II, the build stage. Barrett-Gonzalez explains that “the kinds of safety criteria that we hold to are enforced by rules and regulations, such as the U.S. government’s FAR 23 airworthiness standards that govern small airplanes and FAR 27 standards for smaller helicopters.” That standard of safety, he says, is meant to ensure that the probability of a fatal event is no greater than one in 1 million flight hours. “For larger aircraft like the big Boeing commercial jets and larger helicopters, the standard is even more stringent. It’s one in 1 billion flight hours.”

That focus on safety doesn’t come without a cost, Barrett-Gonzalez adds. “The current thing that is keeping an aircraft like the Mamba from going from the drawing board to reality is that it’s costly. We could do what a Star Wars podracer can do, but that’s a $3.2 million machine. And then, only maybe Elon Musk and half a dozen other people could afford it.”

Several would-be hoverbike manufacturers have enticed potential buyers with price points more than an order of magnitude lower than Barrett-Gonzalez’s estimate. But Barrett-Gonzalez points out that they don’t include the combination of safety features built into the Mamba design. The Mamba has a roll cage, and the motors are cross-shafted. “So if you lose one motor you don’t come spiraling out of the sky,” he says. What’s more, the Mamba’s rotors are arranged according to a patented design that the team says makes it impossible for a rider or bystander to come in contact with the machine’s spinning blades.

For anyone who might argue that the Mamba project imploded because of overdesign, Barrett-Gonzalez recalls the Mamba team having extensive briefings with the director of the U.S. Federal Aviation Administration’s small airplanes directorate. “And he put it plainly: ‘The FAA will not certify a human eggbeater,’” says Barrett-Gonzalez.

“We could do what a Star Wars podracer can do, but that’s a $3.2 million machine. And then, only maybe Elon Musk and half a dozen other people could afford it.” —Ronald Barrett-Gonzalez, University of Kansas

Hover (a hoverbike hopeful formerly known as Hoversurf) recently moved its headquarters from California back to Russia, and Joby Aviation decided to start its electric vertical-takeoff-and-landing (eVTOL) air taxi business in Dubai. These moves might not necessarily indicate their need to generate revenue before refinements to their designs will give them the ability to meet U.S. safety standards. But that explanation is as plausible as any. “Neither Russia nor Dubai have mature airborne safety standards that cover vehicles of this type,” says Barrett-Gonzalez.

Where Are They Now?

In 2014, IEEE Spectrum reported on Aerofex’s pledge to deliver a commercially available hoverbike, the Aero-X, by 2017. Spoiler alert: It didn’t happen. Though Aerofex is still in business, the company retired the Aero-X before the aircraft’s anticipated go-to-market date. The company proudly recalls the progress it made during Aero-X’s development, including kinesthetic control, which lets the pilot stabilize and control a personal aircraft by shifting their weight pretty much the same way one does when riding a bike. But 16 years after its 2008 maiden flight, the $85,000 Aero-X is still not available for sale.

Seven years after its initial go-to-market date, Aerofex’s Aero-X is still not available for sale.Aerofex

Meanwhile, Hover’s series of Scorpion hoverbike designs have gotten plenty of press attention. But as of this writing, the company’s flying motorcycles are still in the preorder stage, with no indication regarding when models like the $150,000 S-3 will be delivered to people who put down deposits.

And Tetra Aviation, the Tokyo startup that won the $100,000 Pratt & Whitney Disruptor Award by the GoFly judges for its Mk-5 single-seat eVTOL vehicle, is also stuck in the development phase. Tetra said it planned to offer the Mk-5, with its 32 vertical lift rotors distributed across long, thin, aluminum-and-carbon-fiber wings and a single pusher prop at the rear, for $320,000, beginning in 2022. But the 8.5-meter wide, 6.1-meter-long machine, which is supposed to travel 160 kilometers (at speeds up to 160 kilometers per hour) on a single charge, is still in the preorder stage.

According to the statements made by the companies seeking to market hoverbikes, the vehicles have been two or three years away for more than a decade. The market predictions made by these companies are starting to sound a lot like an old saw about nuclear fusion, which claims fusion has been “just 20 years away” for nearly 50 years.

Andrew Ng has serious street cred in artificial intelligence. He pioneered the use of graphics processing units (GPUs) to train deep learning models in the late 2000s with his students at Stanford University, cofounded Google Brain in 2011, and then served for three years as chief scientist for Baidu, where he helped build the Chinese tech giant’s AI group. So when he says he has identified the next big shift in artificial intelligence, people listen. And that’s what he told IEEE Spectrum in an exclusive Q&A.

Ng’s current efforts are focused on his company Landing AI, which built a platform called LandingLens to help manufacturers improve visual inspection with computer vision. He has also become something of an evangelist for what he calls the data-centric AI movement, which he says can yield “small data” solutions to big issues in AI, including model efficiency, accuracy, and bias.

Andrew Ng on...

• What’s next for really big models
• The career advice he didn’t listen to
• Defining the data-centric AI movement
• Synthetic data
• Why Landing AI asks its customers to do the work

The great advances in deep learning over the past decade or so have been powered by ever-bigger models crunching ever-bigger amounts of data. Some people argue that that’s an unsustainable trajectory. Do you agree that it can’t go on that way?

Andrew Ng: This is a big question. We’ve seen foundation models in NLP [natural language processing]. I’m excited about NLP models getting even bigger, and also about the potential of building foundation models in computer vision. I think there’s lots of signal to still be exploited in video: We have not been able to build foundation models yet for video because of compute bandwidth and the cost of processing video, as opposed to tokenized text. So I think that this engine of scaling up deep learning algorithms, which has been running for something like 15 years now, still has steam in it. Having said that, it only applies to certain problems, and there’s a set of other problems that need small data solutions.

When you say you want a foundation model for computer vision, what do you mean by that?

Ng: This is a term coined by Percy Liang and some of my friends at Stanford to refer to very large models, trained on very large data sets, that can be tuned for specific applications. For example, GPT-3 is an example of a foundation model [for NLP]. Foundation models offer a lot of promise as a new paradigm in developing machine learning applications, but also challenges in terms of making sure that they’re reasonably fair and free from bias, especially if many of us will be building on top of them.

What needs to happen for someone to build a foundation model for video?

Ng: I think there is a scalability problem. The compute power needed to process the large volume of images for video is significant, and I think that’s why foundation models have arisen first in NLP. Many researchers are working on this, and I think we’re seeing early signs of such models being developed in computer vision. But I’m confident that if a semiconductor maker gave us 10 times more processor power, we could easily find 10 times more video to build such models for vision.

Having said that, a lot of what’s happened over the past decade is that deep learning has happened in consumer-facing companies that have large user bases, sometimes billions of users, and therefore very large data sets. While that paradigm of machine learning has driven a lot of economic value in consumer software, I find that that recipe of scale doesn’t work for other industries.

Back to top

It’s funny to hear you say that, because your early work was at a consumer-facing company with millions of users.

Ng: Over a decade ago, when I proposed starting the Google Brain project to use Google’s compute infrastructure to build very large neural networks, it was a controversial step. One very senior person pulled me aside and warned me that starting Google Brain would be bad for my career. I think he felt that the action couldn’t just be in scaling up, and that I should instead focus on architecture innovation.

“In many industries where giant data sets simply don’t exist, I think the focus has to shift from big data to good data. Having 50 thoughtfully engineered examples can be sufficient to explain to the neural network what you want it to learn.”
—Andrew Ng, CEO & Founder, Landing AI

I remember when my students and I published the first NeurIPS workshop paper advocating using CUDA, a platform for processing on GPUs, for deep learning—a different senior person in AI sat me down and said, “CUDA is really complicated to program. As a programming paradigm, this seems like too much work.” I did manage to convince him; the other person I did not convince.

I expect they’re both convinced now.

Ng: I think so, yes.

Over the past year as I’ve been speaking to people about the data-centric AI movement, I’ve been getting flashbacks to when I was speaking to people about deep learning and scalability 10 or 15 years ago. In the past year, I’ve been getting the same mix of “there’s nothing new here” and “this seems like the wrong direction.”

Back to top

How do you define data-centric AI, and why do you consider it a movement?

Ng: Data-centric AI is the discipline of systematically engineering the data needed to successfully build an AI system. For an AI system, you have to implement some algorithm, say a neural network, in code and then train it on your data set. The dominant paradigm over the last decade was to download the data set while you focus on improving the code. Thanks to that paradigm, over the last decade deep learning networks have improved significantly, to the point where for a lot of applications the code—the neural network architecture—is basically a solved problem. So for many practical applications, it’s now more productive to hold the neural network architecture fixed, and instead find ways to improve the data.

When I started speaking about this, there were many practitioners who, completely appropriately, raised their hands and said, “Yes, we’ve been doing this for 20 years.” This is the time to take the things that some individuals have been doing intuitively and make it a systematic engineering discipline.

The data-centric AI movement is much bigger than one company or group of researchers. My collaborators and I organized a data-centric AI workshop at NeurIPS, and I was really delighted at the number of authors and presenters that showed up.

You often talk about companies or institutions that have only a small amount of data to work with. How can data-centric AI help them?

Ng: You hear a lot about vision systems built with millions of images—I once built a face recognition system using 350 million images. Architectures built for hundreds of millions of images don’t work with only 50 images. But it turns out, if you have 50 really good examples, you can build something valuable, like a defect-inspection system. In many industries where giant data sets simply don’t exist, I think the focus has to shift from big data to good data. Having 50 thoughtfully engineered examples can be sufficient to explain to the neural network what you want it to learn.

When you talk about training a model with just 50 images, does that really mean you’re taking an existing model that was trained on a very large data set and fine-tuning it? Or do you mean a brand new model that’s designed to learn only from that small data set?

Ng: Let me describe what Landing AI does. When doing visual inspection for manufacturers, we often use our own flavor of RetinaNet. It is a pretrained model. Having said that, the pretraining is a small piece of the puzzle. What’s a bigger piece of the puzzle is providing tools that enable the manufacturer to pick the right set of images [to use for fine-tuning] and label them in a consistent way. There’s a very practical problem we’ve seen spanning vision, NLP, and speech, where even human annotators don’t agree on the appropriate label. For big data applications, the common response has been: If the data is noisy, let’s just get a lot of data and the algorithm will average over it. But if you can develop tools that flag where the data’s inconsistent and give you a very targeted way to improve the consistency of the data, that turns out to be a more efficient way to get a high-performing system.

“Collecting more data often helps, but if you try to collect more data for everything, that can be a very expensive activity.”
—Andrew Ng

For example, if you have 10,000 images where 30 images are of one class, and those 30 images are labeled inconsistently, one of the things we do is build tools to draw your attention to the subset of data that’s inconsistent. So you can very quickly relabel those images to be more consistent, and this leads to improvement in performance.

Could this focus on high-quality data help with bias in data sets? If you’re able to curate the data more before training?

Ng: Very much so. Many researchers have pointed out that biased data is one factor among many leading to biased systems. There have been many thoughtful efforts to engineer the data. At the NeurIPS workshop, Olga Russakovsky gave a really nice talk on this. At the main NeurIPS conference, I also really enjoyed Mary Gray’s presentation, which touched on how data-centric AI is one piece of the solution, but not the entire solution. New tools like Datasheets for Datasets also seem like an important piece of the puzzle.

One of the powerful tools that data-centric AI gives us is the ability to engineer a subset of the data. Imagine training a machine-learning system and finding that its performance is okay for most of the data set, but its performance is biased for just a subset of the data. If you try to change the whole neural network architecture to improve the performance on just that subset, it’s quite difficult. But if you can engineer a subset of the data you can address the problem in a much more targeted way.

When you talk about engineering the data, what do you mean exactly?

Ng: In AI, data cleaning is important, but the way the data has been cleaned has often been in very manual ways. In computer vision, someone may visualize images through a Jupyter notebook and maybe spot the problem, and maybe fix it. But I’m excited about tools that allow you to have a very large data set, tools that draw your attention quickly and efficiently to the subset of data where, say, the labels are noisy. Or to quickly bring your attention to the one class among 100 classes where it would benefit you to collect more data. Collecting more data often helps, but if you try to collect more data for everything, that can be a very expensive activity.

For example, I once figured out that a speech-recognition system was performing poorly when there was car noise in the background. Knowing that allowed me to collect more data with car noise in the background, rather than trying to collect more data for everything, which would have been expensive and slow.

Back to top

What about using synthetic data, is that often a good solution?

Ng: I think synthetic data is an important tool in the tool chest of data-centric AI. At the NeurIPS workshop, Anima Anandkumar gave a great talk that touched on synthetic data. I think there are important uses of synthetic data that go beyond just being a preprocessing step for increasing the data set for a learning algorithm. I’d love to see more tools to let developers use synthetic data generation as part of the closed loop of iterative machine learning development.

Do you mean that synthetic data would allow you to try the model on more data sets?

Ng: Not really. Here’s an example. Let’s say you’re trying to detect defects in a smartphone casing. There are many different types of defects on smartphones. It could be a scratch, a dent, pit marks, discoloration of the material, other types of blemishes. If you train the model and then find through error analysis that it’s doing well overall but it’s performing poorly on pit marks, then synthetic data generation allows you to address the problem in a more targeted way. You could generate more data just for the pit-mark category.

“In the consumer software Internet, we could train a handful of machine-learning models to serve a billion users. In manufacturing, you might have 10,000 manufacturers building 10,000 custom AI models.”
—Andrew Ng

Synthetic data generation is a very powerful tool, but there are many simpler tools that I will often try first. Such as data augmentation, improving labeling consistency, or just asking a factory to collect more data.

Back to top

To make these issues more concrete, can you walk me through an example? When a company approaches Landing AI and says it has a problem with visual inspection, how do you onboard them and work toward deployment?

Ng: When a customer approaches us we usually have a conversation about their inspection problem and look at a few images to verify that the problem is feasible with computer vision. Assuming it is, we ask them to upload the data to the LandingLens platform. We often advise them on the methodology of data-centric AI and help them label the data.

One of the foci of Landing AI is to empower manufacturing companies to do the machine learning work themselves. A lot of our work is making sure the software is fast and easy to use. Through the iterative process of machine learning development, we advise customers on things like how to train models on the platform, when and how to improve the labeling of data so the performance of the model improves. Our training and software supports them all the way through deploying the trained model to an edge device in the factory.

How do you deal with changing needs? If products change or lighting conditions change in the factory, can the model keep up?

Ng: It varies by manufacturer. There is data drift in many contexts. But there are some manufacturers that have been running the same manufacturing line for 20 years now with few changes, so they don’t expect changes in the next five years. Those stable environments make things easier. For other manufacturers, we provide tools to flag when there’s a significant data-drift issue. I find it really important to empower manufacturing customers to correct data, retrain, and update the model. Because if something changes and it’s 3 a.m. in the United States, I want them to be able to adapt their learning algorithm right away to maintain operations.

In the consumer software Internet, we could train a handful of machine-learning models to serve a billion users. In manufacturing, you might have 10,000 manufacturers building 10,000 custom AI models. The challenge is, how do you do that without Landing AI having to hire 10,000 machine learning specialists?

So you’re saying that to make it scale, you have to empower customers to do a lot of the training and other work.

Ng: Yes, exactly! This is an industry-wide problem in AI, not just in manufacturing. Look at health care. Every hospital has its own slightly different format for electronic health records. How can every hospital train its own custom AI model? Expecting every hospital’s IT personnel to invent new neural-network architectures is unrealistic. The only way out of this dilemma is to build tools that empower the customers to build their own models by giving them tools to engineer the data and express their domain knowledge. That’s what Landing AI is executing in computer vision, and the field of AI needs other teams to execute this in other domains.

Is there anything else you think it’s important for people to understand about the work you’re doing or the data-centric AI movement?

Ng: In the last decade, the biggest shift in AI was a shift to deep learning. I think it’s quite possible that in this decade the biggest shift will be to data-centric AI. With the maturity of today’s neural network architectures, I think for a lot of the practical applications the bottleneck will be whether we can efficiently get the data we need to develop systems that work well. The data-centric AI movement has tremendous energy and momentum across the whole community. I hope more researchers and developers will jump in and work on it.

Back to top

This article appears in the April 2022 print issue as “Andrew Ng, AI Minimalist.”

The end of Moore’s Law is looming. Engineers and designers can do only so much to miniaturize transistors and pack as many of them as possible into chips. So they’re turning to other approaches to chip design, incorporating technologies like AI into the process.

Samsung, for instance, is adding AI to its memory chips to enable processing in memory, thereby saving energy and speeding up machine learning. Speaking of speed, Google’s TPU V4 AI chip has doubled its processing power compared with that of its previous version.

But AI holds still more promise and potential for the semiconductor industry. To better understand how AI is set to revolutionize chip design, we spoke with Heather Gorr, senior product manager for MathWorks’ MATLAB platform.

How is AI currently being used to design the next generation of chips?

Heather Gorr: AI is such an important technology because it’s involved in most parts of the cycle, including the design and manufacturing process. There’s a lot of important applications here, even in the general process engineering where we want to optimize things. I think defect detection is a big one at all phases of the process, especially in manufacturing. But even thinking ahead in the design process, [AI now plays a significant role] when you’re designing the light and the sensors and all the different components. There’s a lot of anomaly detection and fault mitigation that you really want to consider.

Heather GorrMathWorks

Then, thinking about the logistical modeling that you see in any industry, there is always planned downtime that you want to mitigate; but you also end up having unplanned downtime. So, looking back at that historical data of when you’ve had those moments where maybe it took a bit longer than expected to manufacture something, you can take a look at all of that data and use AI to try to identify the proximate cause or to see something that might jump out even in the processing and design phases. We think of AI oftentimes as a predictive tool, or as a robot doing something, but a lot of times you get a lot of insight from the data through AI.

What are the benefits of using AI for chip design?

Gorr: Historically, we’ve seen a lot of physics-based modeling, which is a very intensive process. We want to do a reduced order model, where instead of solving such a computationally expensive and extensive model, we can do something a little cheaper. You could create a surrogate model, so to speak, of that physics-based model, use the data, and then do your parameter sweeps, your optimizations, your Monte Carlo simulations using the surrogate model. That takes a lot less time computationally than solving the physics-based equations directly. So, we’re seeing that benefit in many ways, including the efficiency and economy that are the results of iterating quickly on the experiments and the simulations that will really help in the design.

So it’s like having a digital twin in a sense?

Gorr: Exactly. That’s pretty much what people are doing, where you have the physical system model and the experimental data. Then, in conjunction, you have this other model that you could tweak and tune and try different parameters and experiments that let sweep through all of those different situations and come up with a better design in the end.

So, it’s going to be more efficient and, as you said, cheaper?

Gorr: Yeah, definitely. Especially in the experimentation and design phases, where you’re trying different things. That’s obviously going to yield dramatic cost savings if you’re actually manufacturing and producing [the chips]. You want to simulate, test, experiment as much as possible without making something using the actual process engineering.

We’ve talked about the benefits. How about the drawbacks?

Gorr: The [AI-based experimental models] tend to not be as accurate as physics-based models. Of course, that’s why you do many simulations and parameter sweeps. But that’s also the benefit of having that digital twin, where you can keep that in mind—it’s not going to be as accurate as that precise model that we’ve developed over the years.

Both chip design and manufacturing are system intensive; you have to consider every little part. And that can be really challenging. It’s a case where you might have models to predict something and different parts of it, but you still need to bring it all together.

One of the other things to think about too is that you need the data to build the models. You have to incorporate data from all sorts of different sensors and different sorts of teams, and so that heightens the challenge.

How can engineers use AI to better prepare and extract insights from hardware or sensor data?

Gorr: We always think about using AI to predict something or do some robot task, but you can use AI to come up with patterns and pick out things you might not have noticed before on your own. People will use AI when they have high-frequency data coming from many different sensors, and a lot of times it’s useful to explore the frequency domain and things like data synchronization or resampling. Those can be really challenging if you’re not sure where to start.

One of the things I would say is, use the tools that are available. There’s a vast community of people working on these things, and you can find lots of examples [of applications and techniques] on GitHub or MATLAB Central, where people have shared nice examples, even little apps they’ve created. I think many of us are buried in data and just not sure what to do with it, so definitely take advantage of what’s already out there in the community. You can explore and see what makes sense to you, and bring in that balance of domain knowledge and the insight you get from the tools and AI.

What should engineers and designers consider when using AI for chip design?

Gorr: Think through what problems you’re trying to solve or what insights you might hope to find, and try to be clear about that. Consider all of the different components, and document and test each of those different parts. Consider all of the people involved, and explain and hand off in a way that is sensible for the whole team.

How do you think AI will affect chip designers’ jobs?

Gorr: It’s going to free up a lot of human capital for more advanced tasks. We can use AI to reduce waste, to optimize the materials, to optimize the design, but then you still have that human involved whenever it comes to decision-making. I think it’s a great example of people and technology working hand in hand. It’s also an industry where all people involved—even on the manufacturing floor—need to have some level of understanding of what’s happening, so this is a great industry for advancing AI because of how we test things and how we think about them before we put them on the chip.

How do you envision the future of AI and chip design?

Gorr: It’s very much dependent on that human element—involving people in the process and having that interpretable model. We can do many things with the mathematical minutiae of modeling, but it comes down to how people are using it, how everybody in the process is understanding and applying it. Communication and involvement of people of all skill levels in the process are going to be really important. We’re going to see less of those superprecise predictions and more transparency of information, sharing, and that digital twin—not only using AI but also using our human knowledge and all of the work that many people have done over the years.

Quantum computing is a devilishly complex technology, with many technical hurdles impacting its development. Of these challenges two critical issues stand out: miniaturization and qubit quality.

IBM has adopted the superconducting qubit road map of reaching a 1,121-qubit processor by 2023, leading to the expectation that 1,000 qubits with today’s qubit form factor is feasible. However, current approaches will require very large chips (50 millimeters on a side, or larger) at the scale of small wafers, or the use of chiplets on multichip modules. While this approach will work, the aim is to attain a better path toward scalability.

Now researchers at MIT have been able to both reduce the size of the qubits and done so in a way that reduces the interference that occurs between neighboring qubits. The MIT researchers have increased the number of superconducting qubits that can be added onto a device by a factor of 100.

“We are addressing both qubit miniaturization and quality,” said William Oliver, the director for the Center for Quantum Engineering at MIT. “Unlike conventional transistor scaling, where only the number really matters, for qubits, large numbers are not sufficient, they must also be high-performance. Sacrificing performance for qubit number is not a useful trade in quantum computing. They must go hand in hand.”

The key to this big increase in qubit density and reduction of interference comes down to the use of two-dimensional materials, in particular the 2D insulator hexagonal boron nitride (hBN). The MIT researchers demonstrated that a few atomic monolayers of hBN can be stacked to form the insulator in the capacitors of a superconducting qubit.

Just like other capacitors, the capacitors in these superconducting circuits take the form of a sandwich in which an insulator material is sandwiched between two metal plates. The big difference for these capacitors is that the superconducting circuits can operate only at extremely low temperatures—less than 0.02 degrees above absolute zero (-273.15 °C).

Superconducting qubits are measured at temperatures as low as 20 millikelvin in a dilution refrigerator.Nathan Fiske/MIT

In that environment, insulating materials that are available for the job, such as PE-CVD silicon oxide or silicon nitride, have quite a few defects that are too lossy for quantum computing applications. To get around these material shortcomings, most superconducting circuits use what are called coplanar capacitors. In these capacitors, the plates are positioned laterally to one another, rather than on top of one another.

As a result, the intrinsic silicon substrate below the plates and to a smaller degree the vacuum above the plates serve as the capacitor dielectric. Intrinsic silicon is chemically pure and therefore has few defects, and the large size dilutes the electric field at the plate interfaces, all of which leads to a low-loss capacitor. The lateral size of each plate in this open-face design ends up being quite large (typically 100 by 100 micrometers) in order to achieve the required capacitance.

In an effort to move away from the large lateral configuration, the MIT researchers embarked on a search for an insulator that has very few defects and is compatible with superconducting capacitor plates.

“We chose to study hBN because it is the most widely used insulator in 2D material research due to its cleanliness and chemical inertness,” said colead author Joel Wang, a research scientist in the Engineering Quantum Systems group of the MIT Research Laboratory for Electronics.

On either side of the hBN, the MIT researchers used the 2D superconducting material, niobium diselenide. One of the trickiest aspects of fabricating the capacitors was working with the niobium diselenide, which oxidizes in seconds when exposed to air, according to Wang. This necessitates that the assembly of the capacitor occur in a glove box filled with argon gas.

While this would seemingly complicate the scaling up of the production of these capacitors, Wang doesn’t regard this as a limiting factor.

“What determines the quality factor of the capacitor are the two interfaces between the two materials,” said Wang. “Once the sandwich is made, the two interfaces are “sealed” and we don’t see any noticeable degradation over time when exposed to the atmosphere.”

This lack of degradation is because around 90 percent of the electric field is contained within the sandwich structure, so the oxidation of the outer surface of the niobium diselenide does not play a significant role anymore. This ultimately makes the capacitor footprint much smaller, and it accounts for the reduction in cross talk between the neighboring qubits.

“The main challenge for scaling up the fabrication will be the wafer-scale growth of hBN and 2D superconductors like [niobium diselenide], and how one can do wafer-scale stacking of these films,” added Wang.

Wang believes that this research has shown 2D hBN to be a good insulator candidate for superconducting qubits. He says that the groundwork the MIT team has done will serve as a road map for using other hybrid 2D materials to build superconducting circuits.