%0 Report %1 northrop2006 %A Northrop, L. %A Feiler, P. %A Gabriel, R. P. %A Goodenough, J. %A Linger, R. %A Longstaff, T. %A Kazman, R. %A Klein, M. %A Schmidt, D. %A Sullivan, K. %A Wallnau, K. %D 2006 %E Pollak, W. %K large scale software %T Ultra-Large-Scale Systems - The Software Challenge of the Future %U http://www.sei.cmu.edu/uls/downloads.html %X What are ULS systems? The U. S. Department of Defense (DoD) has a goal of information dominance—to achieve and exploit superior collection, fusion, analysis, and use of information to meet mission objectives. This goal depends on increasingly complex systems characterized by thousands of platforms, sensors, decision nodes, weapons, and warfighters connected through heterogeneous wired and wireless networks. These systems will push far beyond the size of today’s systems and systems of systems by every measure: number of lines of code; number of people employing the system for different purposes; amount of data stored, accessed, manipulated, and refined; number of connections and interdependencies among software components; and number of hardware elements. They will be ultra-large-scale (ULS) systems. How are ULS systems different? The sheer scale of ULS systems will change everything. ULS systems will necessarily be decentralized in a variety of ways, developed and used by a wide variety of stakeholders with conflicting needs, evolving continuously, and constructed from heterogeneous parts. People will not just be users of a ULS system; they will be elements of the system. Software and hardware failures will be the norm rather than the exception. The acquisition of a ULS system will be simultaneous with its operation and will require new methods for control. These characteristics are beginning to emerge in today’s DoD systems of systems; in ULS systems they will dominate. Consequently, ULS systems will place unprecedented demands on software acquisition, production, deployment, management, documentation, usage, and evolution practices. Challenges of ULS systems Fundamental gaps in our current understanding of software and software development at the scale of ULS systems present profound impediments to the technically and economically effective achievement of the DoD goal of deterrence and dominance based on information superiority. These gaps are strategic, not tactical. They are unlikely to be addressed adequately by incremental research within established categories. Rather, we require a broad new conception of both the nature of such systems and new ideas for how to develop them. We will need to look at them differently, not just as systems or systems of systems, but as socio-technical ecosystems. We will face fundamental challenges in the design and evolution, orchestration and control, and monitoring and assessment of ULS systems. These challenges require breakthrough research. The SEI's ULS research agenda We propose a ULS systems research agenda for an interdisciplinary portfolio of research in at least the following areas: * Human Interaction: involves anthropologists, sociologists, and social scientists conducting detailed socio-technical analyses of user interactions in the field, with the goal of understanding how to construct and evolve such socio-technical systems effectively. * Computational Emergence: explores the use of methods and tools based on economics and game theory (e.g., mechanism design) to ensure globally optimal ULS system behavior and explores metaheuristics and digital evolution to augment the cognitive limits of human designers. * Design: broadens the traditional technology-centric definition of design to include people and organizations; social, cognitive, and economic considerations; and design structures such as design rules and government policies. * Computational Engineering: focuses on evolving the expressiveness of representations to accommodate the semantic diversity of many languages and focuses on providing automated support for computing the evolving behavior of components and their compositions. * Adaptive System Infrastructure: investigates integrated development environments and runtime platforms that will support the decentralized nature of ULS systems as well as technologies, methods, and theories that will enable ULS systems to be developed in their deployment environments. * Adaptable and Predictable System Quality: focuses on how to maintain quality in a ULS system in the face of continuous change, ongoing failures, and attacks and focuses on how to identify, predict, and control new indicators of system health (akin to the U. S. gross domestic product) that are needed because of the scale of ULS systems. * Policy, Acquisition, and Management: focuses on transforming acquisition policies and processes to accommodate the rapid and continuous evolution of ULS systems by treating suppliers and supply chains as intrinsic and essential components of a ULS system. The proposed research does not supplant current, important software research but rather significantly expands its horizons. Moreover, because we are focused on systems of the future, we have purposely avoided couching our descriptions in terms of today’s technology. The envisioned outcome of the proposed research is a spectrum of technologies and methods for developing these systems of the future, with national-security, economic, and societal benefits that extend far beyond ULS systems themselves. Though our research agenda does not prescribe a single, definitive roadmap, we offer three structures that suggest ways to cluster and prioritize groups of research areas mapping the research areas and topics to (1) specific DoD missions and required capabilities, (2) DoD research funding types required to support them, and (3) estimates of the relative starting points of the research. These structures can then be used to define one or more roadmaps that could lead to one or more ULS systems research programs or projects. Recommendations As a first step, we recommend the funding and establishment of a ULS System Research Startup Initiative, which over the course of the next two years would, among other things, * work with others to conduct new basic research in key areas; * foster the growth of a community of informed stakeholders and researchers; and * formulate and issue an initial Broad Agency Announcement (BAA) to attract researchers with proven expertise in the diverse set of disciplines (e.g., software engineering, economics, human factors, cognitive psychology, sociology, systems engineering, and business policy) that are collectively required to meet the challenge of ULS systems. The United States needs a program that will fund the software research required to sustain ongoing transformations in national defense and achieve the DoD goal of information dominance. The key challenge is the decision to move forward. The ULS System Research Agenda presented in Ultra-Large-Scale Systems: The Software Challenge of the Future provides the starting point for the path ahead.

@techreport{northrop2006, abstract = {What are ULS systems? The U. S. Department of Defense (DoD) has a goal of information dominance—to achieve and exploit superior collection, fusion, analysis, and use of information to meet mission objectives. This goal depends on increasingly complex systems characterized by thousands of platforms, sensors, decision nodes, weapons, and warfighters connected through heterogeneous wired and wireless networks. These systems will push far beyond the size of today’s systems and systems of systems by every measure: number of lines of code; number of people employing the system for different purposes; amount of data stored, accessed, manipulated, and refined; number of connections and interdependencies among software components; and number of hardware elements. They will be ultra-large-scale (ULS) systems. How are ULS systems different? The sheer scale of ULS systems will change everything. ULS systems will necessarily be decentralized in a variety of ways, developed and used by a wide variety of stakeholders with conflicting needs, evolving continuously, and constructed from heterogeneous parts. People will not just be users of a ULS system; they will be elements of the system. Software and hardware failures will be the norm rather than the exception. The acquisition of a ULS system will be simultaneous with its operation and will require new methods for control. These characteristics are beginning to emerge in today’s DoD systems of systems; in ULS systems they will dominate. Consequently, ULS systems will place unprecedented demands on software acquisition, production, deployment, management, documentation, usage, and evolution practices. Challenges of ULS systems Fundamental gaps in our current understanding of software and software development at the scale of ULS systems present profound impediments to the technically and economically effective achievement of the DoD goal of deterrence and dominance based on information superiority. These gaps are strategic, not tactical. They are unlikely to be addressed adequately by incremental research within established categories. Rather, we require a broad new conception of both the nature of such systems and new ideas for how to develop them. We will need to look at them differently, not just as systems or systems of systems, but as socio-technical ecosystems. We will face fundamental challenges in the design and evolution, orchestration and control, and monitoring and assessment of ULS systems. These challenges require breakthrough research. The SEI's ULS research agenda We propose a ULS systems research agenda for an interdisciplinary portfolio of research in at least the following areas: * Human Interaction: involves anthropologists, sociologists, and social scientists conducting detailed socio-technical analyses of user interactions in the field, with the goal of understanding how to construct and evolve such socio-technical systems effectively. * Computational Emergence: explores the use of methods and tools based on economics and game theory (e.g., mechanism design) to ensure globally optimal ULS system behavior and explores metaheuristics and digital evolution to augment the cognitive limits of human designers. * Design: broadens the traditional technology-centric definition of design to include people and organizations; social, cognitive, and economic considerations; and design structures such as design rules and government policies. * Computational Engineering: focuses on evolving the expressiveness of representations to accommodate the semantic diversity of many languages and focuses on providing automated support for computing the evolving behavior of components and their compositions. * Adaptive System Infrastructure: investigates integrated development environments and runtime platforms that will support the decentralized nature of ULS systems as well as technologies, methods, and theories that will enable ULS systems to be developed in their deployment environments. * Adaptable and Predictable System Quality: focuses on how to maintain quality in a ULS system in the face of continuous change, ongoing failures, and attacks and focuses on how to identify, predict, and control new indicators of system health (akin to the U. S. gross domestic product) that are needed because of the scale of ULS systems. * Policy, Acquisition, and Management: focuses on transforming acquisition policies and processes to accommodate the rapid and continuous evolution of ULS systems by treating suppliers and supply chains as intrinsic and essential components of a ULS system. The proposed research does not supplant current, important software research but rather significantly expands its horizons. Moreover, because we are focused on systems of the future, we have purposely avoided couching our descriptions in terms of today’s technology. The envisioned outcome of the proposed research is a spectrum of technologies and methods for developing these systems of the future, with national-security, economic, and societal benefits that extend far beyond ULS systems themselves. Though our research agenda does not prescribe a single, definitive roadmap, we offer three structures that suggest ways to cluster and prioritize groups of research areas mapping the research areas and topics to (1) specific DoD missions and required capabilities, (2) DoD research funding types required to support them, and (3) estimates of the relative starting points of the research. These structures can then be used to define one or more roadmaps that could lead to one or more ULS systems research programs or projects. Recommendations As a first step, we recommend the funding and establishment of a ULS System Research Startup Initiative, which over the course of the next two years would, among other things, * work with others to conduct new basic research in key areas; * foster the growth of a community of informed stakeholders and researchers; and * formulate and issue an initial Broad Agency Announcement (BAA) to attract researchers with proven expertise in the diverse set of disciplines (e.g., software engineering, economics, human factors, cognitive psychology, sociology, systems engineering, and business policy) that are collectively required to meet the challenge of ULS systems. The United States needs a program that will fund the software research required to sustain ongoing transformations in national defense and achieve the DoD goal of information dominance. The key challenge is the decision to move forward. The ULS System Research Agenda presented in Ultra-Large-Scale Systems: The Software Challenge of the Future provides the starting point for the path ahead.}, added-at = {2007-08-05T19:17:06.000+0200}, author = {Northrop, L. and Feiler, P. and Gabriel, R. P. and Goodenough, J. and Linger, R. and Longstaff, T. and Kazman, R. and Klein, M. and Schmidt, D. and Sullivan, K. and Wallnau, K.}, biburl = {https://www.bibsonomy.org/bibtex/253c6e83e1f7ec47b378721a81977c8e8/wnpxrz}, editor = {Pollak, W.}, institution = {Software Engineering Institute, Carnegie Mellon}, interhash = {410c990a2cf3a6256a37ea03a00f6826}, intrahash = {53c6e83e1f7ec47b378721a81977c8e8}, keywords = {large scale software}, month = {June}, timestamp = {2007-08-05T19:17:06.000+0200}, title = {{Ultra-Large-Scale Systems - The Software Challenge of the Future}}, url = {http://www.sei.cmu.edu/uls/downloads.html}, year = 2006 }