CARCHITECTURE
Making Automotive Software Safe

Automobiles today can have a hundred or more onboard computers spread across many integrated systems. Besides the mechanical and electronic hardware involved, there are thousands of electronic circuits and millions of lines of software code to develop, run, monitor, debug and safeguard. Should we be worried about the vehicles we drive?

Software's Gone Viral
"Software is the world's most critical industry and will be for years to come," says Mike Devlin, founder, Rational Software, and former IBM general manager. "Most companies are spectacularly unprepared to create the software that will redefine how they interact with customers or that will help deliver their goods and services in new ways."

According to leading automotive information technology (IT) and software engineering firms, such as the Fraunhofer Institute for Experimental Software Engineering (IESE), Horst-Gortz Institute and Robert Bosch GmbH, software is expected to continue to permeate vehicle systems. They say the cost of embedded IT content and associated electronics will exceed 50 percent of new car manufacturing costs by 2015.

Making Software Safe is the Greatest Challenge Ahead

Preventing failures and protecting electrical / electronic automotive IT systems is the greatest challenge automotive developers will face this decade. Click here to learn why. (Video - Fraunhofer IESE; Image — Delphi Corp.)

A number of factors are driving these innovations: time to market reduction, cost optimization, quality improvement, safety conformance, passenger comfort, performance, environmental regulation compliance and vehicle intra- and interconnectivity. While they add functionality, they also increase complexity on embedded systems, as well as the need to ensure their security. Autmotive IT — the sofware, hardware, embedded hardware and firmware — monitors and manages vehicle systems. It must be robust, reliable and safe; hence, they are tested for mechanical, electrical, electromagnetic radiation, vibration and other sources of failure. Every few milliseconds, control software logic analyzes hundreds of inputs from the systems they monitor.

Software quality must to be part of the design
Any new technology can have glitches. Consider your home or office computer. When a new product, operating system or major piece of software is launched in the marketplace, there are often bugs and glitches that arise after a product's release. These bugs were not caught by the software developers and designers during design or development; they are usually fixed via downloadable reprogramming software patches. In addition, computers can suffer software crashes. Unfortunately, when glitches, bugs and crashes happen to automobiles, the consequences are not only expensive, but they are devastating

One metric that measures the size and complexity of a software program is 'source lines of code' (SLOC). Manfred Broy, a leading expert on automotive software and a professor at the Department of Informatics at the Technical University of Munich, Germany, offers these reference points:

• Software is pervasive. Although cars had about 1 million SLOC in 1990, current vehicles can have more than 100 electronic control modules with 100 million SLOC to control over 2,000 functions. Within five years, this will approach 300 million lines. By comparison, the new Boeing 787 Dreamliner employs 6.5 million lines to run its avionics system alone. The Windows XP operating system, which runs many personal computers, has approximately 40 million SLOC.
• Software recalls will become more frequent. A conservative acceptable defect density rate is 0.4 defects per thousand lines of code. If just 5 percent of those are exploitable, it results in 2,000 exploitable bugs for every 100 million SLOC. Catching all of these bugs before vehicles are launched just isn't going to happen.
• Cost matters. Software and related electronics account for 35 percent of total manufacturing costs today; within five years, more than 50 percent of costs will be software-related. At a conservative cost of $10 per line of software code, development of a new vehicle platform reaches $1 billion. That's before any recalls for reprogramming.
• Grammatically correct software may not run as intended. While software code may work from a "grammar and punctuation" perspective, there's no guarantee that what the software was intended to do will happen every time - just as words spelled correctly and sentences punctuated appropriately on a page still may not tell a story.

Looking Beyond Hardware

Computers control vehicle systems. For example, Toyota's Electronic Control Module (ECM) is dedicated to all underhood and powertrain functions, including throttle control. Click here to view how faults, obstructions, outside electrical interference and other threats are safeguarded by fail-safes. However, automakers can not be certain that the software in a vehicle's operating system or the hard-coded software within control computers is sound, nor can they ignore what drivers will do in different situations. (Video / Image  — Toyota Motor Sale

Recent innovation now enables any automotive hardware to be virtually cloned. This enables software developers to write automotive software code — from operating systems to device drivers to interrupt handlers to application programs — without any real hardware being in place. As a result, programmers today have to debug and test software code, for both hardware and software.

The increased workload and time-to-market pressures is not a good mix. "Fixing failures means first finding them," notes Jim Turley, an independent technology analyst and former editor of Embedded Systems Design and Microprocessor Report. Software developers spend nearly 40 percent of their time debugging (what we call that reprogramming) — more than any other task, be it programming, writing specifications or engineering.

The challenge is finding software errors in the midst of millions of lines of code before vehicles are ever sold, or as soon as possible afterwards. "Debugging is equal parts art and science: The science is knowing how to fix flaws, while the art is knowing where to look," Turley adds.

IT security often lags software development and release
In the past, paralleling the personal computer (PC) industry, most vehicle IT systems did not include security functions, as there was little incentive for malicious manipulation. That has changed. "Ninety percent of automotive innovations are IT-based, as are the majority of bugs," adds Christof Parr, director of the Horst-Gortz Institute for IT Security. "These are embedded into automotive systems, some of which have the computing power of current PCs."

"During the past 10 years, the number of automaker recalls has quadrupled," says Dieter Rombach, director of Fraunhofer IESE. He insists that software integrity — from design to reliable, safe functionality in the marketplace — is critical to the industry's success. "Electronic systems and software are the major causes of this."

He cites several drivers for this: Rising new vehicle values; high replacement expense associated with easily stolen or counterfeited components and systems; warranty issues from undetected post-sale chip-tuning and other unauthorized software modifications of vehicle functions; growing concern for privacy and in-car data protection; and emerging new business revenue opportunities that add more wireless vehicle functionality.

Like tother industries, IT security within automotive domain has unfortunately lagged product launches. Rombach, Paar and others in the IT security field insist that much of the software and hardware in current cars has yet to be adequately protected against serious threats, such as manipulation, malfunction or malicious attacks.

Potential attackers include those who have physical or remote access to an automotive device. They focus on trying to identify the secret encryption key, either by direct or indirect methods. Once knowledge of the key is obtained, the device can then be readily manipulated and/or cloned. Sadly, attackers all too often turn out to be vehicle owners, maintenance personnel, competitors and other organizations that dishonestly and illegitimately monitor and/or modify functionality for personal gain.

Direct attacks, such as reverse engineering, attempt to read the cryptographic keys directly from the embedded device's various memories. Unlike conventional automotive reverse engineering, the recovery of just a single cryptographic key — only 16 bytes long or less — is often all that is needed to be successful.

Making Software Safe is the IT Security Enablers

As computers have become more ubiquitous, increased levels of embedded IT security have begun to permeate automobile systems. Click here to view some of the innovations fueling the growth of embedded software and IT security. (Photo - Daimler AG)

It's like finding a needle in a mulitdimensional haystack
IT security techniques guard against unauthorized manipulation or unintended consequences of IT-systems. To be effective, they must physically safe, technically reliable and immune to attacks. Inexpensive and primitive technologies dominate current IT security strategies, but more sophisticated strategies and security tools are beginning to be incorporated into automotive applications.

The development of strong embedded security solutions is not an easy road to travel. Automotive IT and security design face computational, memory limits, cost and other tight constraints. The manufacturing supply chain — OEMs and their multiple layers of suppliers and toolmakers — requires an efficient exchange of design information, as each has implications that must be factored in. For example, who designs the security architecture and most importantly, who has control over the cryptographic keys and other IT security measures become very relevant. In addition, differences between automakers, proprietary boundaries, cumulative tolerance stack-up of assembled components and extreme operating environments challenge progress.

IT security design also differs fundamentally from the engineering of most other technical automotive systems. In conventional automotive systems, a single non-optimum component will not normally invalidate the entire system. For software and IT security solutions, design integrity across and between systems is crucial. A single, seemingly minor flaw in IT security system design — a single line of software code out of the millions of lines in today's automobiles — might not affect the local system. But if not identified before or after launch, that bug could then render programming or security solutions in interconnected systems or controllers insecure, unreliable or worse.

Shifting mindsets: Open standards can build safer systems
Every automaker faces this dilemma across the globe, regardless of whatever market they build or sell in. Individual approaches by OEMs must mature into industry-wide open standards and regulations that motivate common use by all OEMs. Going to market with imperfect platforms and then trying to resolve problems with reprogramming patches that aren't guaranteed to fix problems is becoming untenable, if not irresponsible. In addition, regulatory bodies such as the National Highway Transportation Safety Administration (NHTSA) need to ramp up their engineering and technical expertise, a shortcoming the agency recently acknowledged in writing to the U.S. Transportation Secretary Ray LaHood.

Toyota's approach is typical of many automakers. "We take a holistic approach, looking at standards for whatever regional locations that we're in, as well as any international standards that may exist," says Kristen Tabar, general manager for Electronic System 2 Department at the Toyota Technical Center. "In cases where those standards or regulations are lacking, we even go beyond that and create our own test standards, based on our know-how or experience in the real world environment that the vehicle will see."

Standards Empower Safer Applications

Recently, a movement to open standards backed by sophisticated automotive IT security protocols is enabling new applications that improve safety, performance and other measures. Click here to view some of the emerging innovations. (Photo - General Motors)

Standards are poised to play a significant role in the evolution of automotive functionality and vehicle-to-vehicle connectivity. While comprehensive, global software standards and security processes do not yet exist across the board for automakers in any country, let alone on a global scale, the momentum is building. Manufacturing, servicing, monitoring by regulators (for compliance and public safety purposes) and consumers all stand to benefit.

If automotive source code was open and available to anyone, leveraging open source software for both vehicle development and IT security would enable the collective community to assist in discovering bugs, augmenting the efforts and expertise of the much smaller population of automaker and other personnel currently on the watch. In addition, open standards would enable software to operate across different OEM platforms. Granted, sharing what some OEMs currently consider to be proprietary intellectual property with competitors would require a change in mindsets, but the collaboration would reduce development costs for common software applications for all, as well as those downstream. Importantly, IT security measures would still provide the necessary protections and safeguards.

Inviting, responding to and incorporating user experience (UX) into the design equation is another change agent that is beginning to gain traction. Looking at software and security issues from a UX point-of-view, rather than just from the vehicle system's viewpoint, addresses an obvious design gap: It is human operators that interact with vehicle sensors, actuators and computer modules.

At best, even if software systems and security reliability were to approach being 100 percent error-free, perfection would remain unreachable. Despite all the automatic mechanical, electronic and other fail-safes built into systems, people are the ultimate backup. Proponents say that integrating how people behave in real-world situations into design would go a long ways to realizing safer and more reliable performance. While average or even a range of typical behaviors can be designed in, it is the exceptions outside the norms that are more difficult to account for and manage.

"Providing failure safety in electrical and electronic automotive systems is the greatest challenge for automotive developers in this decade," Rombach stresses. Software will continue to become increasingly complex and require continual updates. The wireless Web, ongoing electrification of the automobile and changing software universe will continue to have far-reaching implications on vehicles and those who build and maintain them. Already we are seeing the spectre that could ensue. "Many recalls today are software related," he continues. "That trend will continue unabated in coming years unless we act now."

(Editor's Note: Bob Chabot is a freelance writer, editor and e-producer for Techs4Tomorrow, MOTOR Magazine, AutoInc., NASTF and other custom publications.)