Summary

Aurora is a citizen of the digital world. She is threatened. The digital systems that surround her are increasingly able to make autonomous decisions over and above her and on her behalf. She feels that her moral rights, as well as the social, economic and political spheres, can be affected by the behavior of such systems. Although unavoidable, the digital world is becoming uncomfortable and potentially hostile to her as a human being and as a citizen. Notwithstanding the introduction of the GDPR and of initiatives to establish criteria on software transparency and accountability, Aurora feels vulnerable and unprotected.
EXOSOUL will build a software personalized exoskeleton that enhances and protects Aurora by mediating her interactions with the digital world according to her own ethics of actions and privacy of data. The exoskeleton disallows or adapts the interactions that would result in unacceptable or morally wrong behaviors according to the ethics and privacy preferences of Aurora. With her software shield, Aurora will feel empowered and in control, and more in balance of forces with the other actors of the digital world.
To reach the breakthrough result of automatically building a personalized exoskeleton, EXOSOUL will address multidisciplinary challenges never touched before: (i) defining the scope for and inferring citizens ethical preferences; (ii) treating privacy as an ethical dimension managed through the disruptive notion of active data; and (iii) automatically synthesizing ethical actuators, i.e., connector components that mediate the interaction between the user and the digital world to enforce her ethical preferences. EXOSOUL will deliver the first concrete contribution to an ethical approach to regulate the digital world in line with the goals of the European Data Protection Supervisor strategy 2015-2019.

Project

Exosoul is an overarching project funded by the University of L’Aquila. The project is about building a software personalized exoskeleton that enhances and protects human beings by mediating their interactions with the digital world according to their own ethics of actions and privacy of data.

Motivation – In their ordinary life, citizens in the digital world continuosly interact with software systems, e.g., by using a mobile device or from on board of a (autonomous) car. These systems are increasingly autonomous in making decisions over and above the users or on behalf of them. Often, their autonomy exceeds the system boundaries and invades user prerogatives. As a consequence, ethical issues – privacy ones included (e.g., unauthorized disclosure and mining of personal data, access to restricted resources) – are emerging as matters of utmost concern since they impact on the moral rights of each human being and affect the social, economic, and political spheres [
1EDPS. Leading by example, The EDPS Strategy 2015-2019. //edps.europa.eu/sites/edp/files/publication/15-07-30_strategy_2015_2019_update_en.pdf, 2015.
], [
2J. P. Burgess, L. Floridi, A. Pols, and J. van den Hoven. Towards a digital ethics – edps ethics advisory group. //edps.europa.eu/sites/edp/files/publication/18-01-25_eag_report_en.pdf, 2018.
], [
3European Group on Ethics in Science and New Technologies. statement on artificial intelligence, robotics and ‘autonomous’ systems. //ec.europa.eu/research/ege/pdf/ege_ai_statement_2018.pdf, 2018.
], [
4Inverardi, P. (2019). The European perspective on responsible computing. Communications of the ACM, 62(4), 64-64
], [
5Autili, M., Di Ruscio, D., Inverardi, P., Pelliccione, P., & Tivoli, M. (2019). A Software Exoskeleton to Protect and Support Citizen’s Ethics and Privacy in the Digital World. IEEE Access, 7, 62011-62021.
]. Besides the philosophical aspects, the way to approach these problems is twofold: regulatory and technical. Europe has recently introduced the GDPR legislation for data protection [
6European Commission. General Data Protection Regulation, 2018.
], Sweden and Germany are at the forefront on the regulation of autonomous vehicles, while a common EU approach to liability rules and insurance for connected and autonomous vehicles is under discussion [
7T. Evas, C. Rohr, F. Dunkerley, and D. Howarth. A common eu approach to liability rules and insurance for connected and autonomous vehicles. DOI: 10.2861/282501, March 28, 2018.
]. The scientific community and some big companies are proposing initiatives to identify problems and establish criteria to develop algorithms and systems that embed autonomous capabilities [
8Association for Computing Machinery US Public Policy Council (USACM). Statement on algorithmic transparency and accountability. //www.acm.org/binaries/content/assets/public-policy/2017_usacm_statement_algorithms.pdf, 2018.
], [
9Partnership on AI. //www.partnershiponai.org/, 2018.
], [], [
11J. L. et al. When computers decide: European recommendations on machine-learned automated decision making. //www.acm.org/binaries/content/assets/public-policy/ie-euacm-adm-report-2018.pdf, 2018.
]. As a matter of fact, the digital world is being recognized as potentially hostile to citizens. The initiatives proposed so far go in the direction to make the world less hostile by introducing new laws, from the regulatory side, and transparency and accountability criteria in software development, from the technical side. Regulation is important as well as in-depth insights into the technology. However, we are fully aware that achieving full adherence to regulation and transparency criteria is very difficult or even impossible in practice. We are facing a paradox: human beings are recognized as central actors, the sensitive targets; but they are passive consumers in the digital world, and the power and the burden to preserve their rights remain in the hands of the (software-) systems producers. In the mangrove societies – Floridi’s powerful metaphor
2In the digital world it is impossible to distinguish whether we are online or offline – instead we are onlife, as it is impossible to understand whether the water in the estuary – where the river meets the sea – is sweet or salty – Ref. L. Floridi. Soft ethics and the governance of the digital. Philosophy & Technology, 31(1):1–8, Mar 2018.
– human beings are unprotected in their interactions with the digital world. The great challenge, unattempted so far, is to comprehensively empower them.

The vision – The goal of EXOSOUL is to equip humans with an automatically generated exoskeleton, a software shield that protects them and their personal data via the mediation of all interactions with the digital world that would result in unacceptable or morally wrong behaviors according to their ethical and privacy preferences. The exoskeleton can take a whole spectrum of forms: from customized soft-libraries that the individual may deploy on the machines being used, to a sophisticated software interface that an individual may “wear”, eventually deployed on a body chip. Empowering the users with a personalized exoskeleton will introduce more symmetry of power in the present digital world and will effectively put humans in the center. Exoskeletons development also opens unprecedented business opportunities in the same way open source software did, which promoted the ethical principles of free software against the monopoly proprietary software producers. The European Union (EU) with its companies can become the scientific and technological leader of the future user-driven privacy and ethics systems. Furthermore, bringing back to the user part of the (digital) control helps to solve liability issues in autonomous systems by readdressing responsibility to users according to their specified ethics.

The challenges ahead – We address the challenge of automatically synthesizing a software exoskeleton starting from the ethics and privacy preferences of the user. In the ethical sphere, this requires to answer several cutting edge research questions concerning the need to: (i) identify a space of ethics and privacy preferences for users, to assess their compatibility with regulations, and to orchestrate interactions of users endorsing different preferences, so as to prevent deadlocks and to promote best ethical practices in digital societies; (ii) infer ethics and privacy preferences from the user, given that neither a person nor a society apply moral categories separately, rather everyday morality is in constant flux among norms, utilitarian assessment of consequences, and evaluation of virtues. We define the exoskeleton by considering two specific classes of interactions that citizens have with the digital world. The first one concerns interactions that involve the exchange of personal data, and that as such impact the privacy dimension, notably interactions with mobile apps through mobile devices. Until now, data are considered as passive entities and the logic implementing their life-cycle is decoupled from the data itself. For each datum that is shared over the Internet, the owner loses its track and control [
12B. Krishnamurthy and C. E. Wills. Characterizing privacy in online social networks. In Proceedings of the First Workshop on Online Social Networks, WOSN ’08, pages 37–42, New York, NY, USA, 2008. ACM.
]. Such problems have been mitigated by means of regulatory (e.g., GDPR) and technical attempts. Unfortunately, these attempts solve the mentioned issues only partially. We propose a disruptive approach that changes the passive nature of data by introducing active data. As part of the exoskeleton, active data encapsulate data with mechanisms that govern their creation, destruction, use, and sharing according to the owner ethical preferences. Destruction is the basic means to provide the right to be forgotten, which requires to equip data with an apoptosis mechanism
3In biological terms, apoptosis, also called programmed cell death, is a mechanism that allows cells to self-destruct when stimulated by the appropriate trigger, internal or external to the cell – ref. Encyclopædia Britannica.
– synthesized from the user’s ethical and privacy preferences – whose enactment depends on the use the digital world makes of the data, beyond parameters like time. The second one concerns the interaction with systems that are equipped with some degree of autonomy and that a user may want to ethically control to some extent. Autonomous vehicles and the so-called trolley problem represent a well-known limiting case, but other more ordinary cases exist [
13G. Contissa, F. Lagioia, and G. Sartor. The ethical knob: ethically-customisable automated vehicles and the law. Artificial Intelligence and Law, 25(3):365–378, 2017.
]. As part of the exoskeleton, we will address the challenge of synthesizing, out of the user’s ethical preferences, an ethical actuator able to intercept the interactions between the autonomous engine and the machine actuators and to prevent behaviours that are not admissible by the ethical preferences. Since this approach cannot be independent from the software the citizens are interacting with, by-product results of the project will be requirements on the way the digital world needs to conform in order to interact with exoskeletons. This is as important as developing the shield since it establishes architecture and protocol requirements the systems producers need to comply with. EXOSOUL citizens will interact only with the part of the digital world that accepts their requirements. This breaks the monopoly of producers by introducing symmetry in the producer/user roles and new economic drivers in the digital market. At the same time, producers can be relieved from the liability burden by readdressing responsibility to users.

Our features

Ethics

Defining the scope for and inferring citizens ethical preferences

Automation

Automatically synthesizing ethical actuators

Privacy

Treating privacy as an ethical dimension managed through the disruptive notion of active data

Research Themes

Logic theories and innovative mechanisms for inferring and specifying privacy and ethical user preferences

To address the challenge of specifying and inferring soft ethical preferences, we will start investigating a kind of “functional morality” [1], which enables machines to autonomously assess and respond to moral challenges. Our own work has addressed various  hard  ethics  problems  on  human  interactions  with  AI,  robotic  and  bionic  systems  [2, 3, 4, 5], concerning the analysis of conflicts between competing normative ethics approaches and the development of public ethical policies to defuse those conflicts.
In operative terms, we will consider the relevant legislation of the member states (e.g., GDPR //eugdpr.org/), ethical reference groups (//edps.europa.eu/sites/edp/files/publication/18-01-25_eag_report_en.pdf//ec.europa.eu/research/ege/pdf/ege_ai_statement_2018.pdf ), the normative approaches to ethics and the European perspective on responsible computing [6]. Furthermore, we will elicit patterns for specifying privacy and ethics out of existing privacy and ethical rules defined by both the academic and industrial communities, examples of which maybe found in our previous work [7].
We will employ an iterative approach to the design and validation of the innovative mechanisms  for  inferring  and  specifying  ethical  and  privacy  preferences. Representative users will be in the loop at every stage. 

References

[1] W. Wallach and C. Allen. Moral Machines: Teaching Robots Right from Wrong. Oxford University Press, Inc., New York, NY, USA, 2010. [BIBTEX]

[2] D. Amoroso and G. Tamburrini. The ethical and legal case against autonomy in weapons systems. Global Jurist, 17, 01 2017. [BIBTEX]

[3] G. Tamburrini. On the ethical framing of research programs in robotics. AI Soc., 31(4):463–471, Nov. 2016. [BIBTEX]

[4] A. Bicchi and G. Tamburrini. Social robotics and societies of robots. The Information Society, 31(3):237–243, 2015. [BIBTEX]

[5] M. Santoro, D. Marino, and G. Tamburrini. Learning robots interacting with humans: from epistemic risk to responsibility. AI & SOCIETY, 22(3):301–314, Jan 2008. [BIBTEX]

[6] Paola Inverardi. 2019. The European perspective on responsible computing. Commun. ACM 62, 4 (March 2019), 64-64. DOI: //doi.org/10.1145/3311783. [BIBTEX]

[7] M. Autili, L. Grunske, M. Lumpe, P. Pelliccione, and A. Tang. Aligning qualitative, real-time, and probabilistic property specification patterns using a structured english grammar. IEEE Transactions on Software Engineering, 41(7):620–638, July 2015. [BIBTEX]

Exoskeleton design and newfangled techniques and tools for managing its life-cycle

This research theme concerns the definition of the exoskeleton software architecture and of the run-time analysis mechanisms, such as monitoring and enforcement, that serve to control the exoskeleton behavior according to the specified privacy and ethics preferences. An exoskeleton is composed of two parts:  active data and ethical actuator.
Active data wrap personal data by adding the logic required to access personal data and manage their  life-cycle, from  creation to destruction, sharing and usage, according  to  the  specified  privacy preferences. The conformance to the privacy preferences is guaranteed by a monitoring and enforcing component that makes use of the internal operations and continuously checks and updates the life-cycle status to promptly detect and correct problems before privacy-violating actions are performed.
The ethical actuator translates conceptual ethical principles into concrete statements that serve as the basis for ethical decision making.  An ethical actuator is composed of:  (i) ethical rules defined by users, (ii) a monitor, (iii) an enforcer, and (iv) ethical actions.

References

Zhang, P., Pelliccione, P., Leung, H., & Li, X. (2018). Automatic generation of predictive monitors from scenario-based specifications. Information and software technology, 98, 5-31. [BIBTEX]

Gian Luca Scoccia, Stefano Ruberto, Ivano Malavolta, Marco Autili, Paola Inverardi: An investigation into Android run-time permissions from the end users’ perspective. MOBILESoft@ICSE 2018: 45-55. [BIBTEX]

Gian Luca Scoccia, Ivano Malavolta, Marco Autili, Amleto Di Salle, Paola Inverardi: User-centric Android flexible permissions. ICSE (Companion Volume) 2017: 365-367. [BIBTEX]

A. Tang, P. Pelliccione, P. Lago, H. Muccini and I. Malavolta, “What Industry Needs from Architectural Languages: A Survey” in IEEE Transactions on Software Engineering, vol. 39, no. 06, pp. 869-891, 2013. [BIBTEX]

Rebekka Wohlrab, Ulf Eliasson, Patrizio Pelliccione, Rogardt Heldal (2019)  Improving the Consistency and Usefulness of Architecture Descriptions: Guidelines for Architects In: IEEE International Conference on Software Architecture (ICSA 2019), Hamburg, Germany, March 25-29. [BIBTEX]

Marco Autili, Paola Inverardi, Massimo Tivoli: Choreography Realizability Enforcement through the Automatic Synthesis of Distributed Coordination Delegates. Sci. Comput. Program. 160: 3-29 (2018). [BIBTEX]

Pelliccione, P., Knauss, E., Heldal, R., Ågren, S. M., Mallozzi, P., Alminger, A., & Borgentun, D. (2017). Automotive architecture framework: The experience of volvo cars. Journal of systems architecture, 77, 83-100. [BIBTEX]
 

Marco Autili, Davide Di Ruscio, Amleto Di Salle, Paola Inverardi, Massimo Tivoli: A Model-Based Synthesis Process for Choreography Realizability Enforcement. FASE 2013: 37-52. [BIBTEX]

Exoskeleton synthesis

This research theme concerns the definition and the realization of automated synthesis methods for the generation of:  (i) a domain-independent exoskeleton starting from the user’s ethical and privacy preferences, and (ii) a domain-specific specialization of the domain-independent exoskeleton from the inputs provided by domain experts.  These inputs regard information that are required to produce the code of the specialized exoskeleton, and package it as required by the target execution environment.
This is  extremely  challenging  since  it  has  to  cope  with  the  complexity  of representing  and  enforcing  ethical and privacy rules. However,  we can base on our  expertise  on the  prevention  of  interaction  mismatches.   Indeed,  in  our  previous  work  [1,  2,  3,  4,  5,  6, 7]  we exploited architectural specifications, including interaction and communication patterns, APIs, etc., to  automatically  generate  integration  and  coordination  code  for  the  components  forming  a  target distributed system.

 
References
 
[1] M. Autili, P. Inverardi, F. Mignosi, R. Spalazzese, and M. Tivoli. Automated synthesis of application-layer connectors from automata-based specifications. In 9th Int. Conf. on Language and Automata Theory and Applications LATA, pages 3–24, 2015. [BIBTEX]
 
[2] M. Autili, P. Inverardi, and M. Tivoli. Automated synthesis of service choreographies. IEEE Software, 32(1):50–57,2015. [BIBTEX]
 
[3] M. Autili, P. Inverardi, and M. Tivoli. Choreography realizability enforcement through the automatic synthesis of distributed coordination delegates. Science of Computer Programming, 160:3 – 29, 2018. [BIBTEX]
 
[4] M. Autili, L. Mostarda, A. Navarra, and M. Tivoli. Synthesis of decentralized and concurrent adaptors for correctly assembling distributed component-based systems. Journal of Systems and Software, 81(12):2210–2236, 2008.[BIBTEX]
 
[5] P. Inverardi and M. Tivoli. Automatic synthesis of modular connectors via composition of protocol mediation patterns. In 2013 35th International Conference on Software Engineering (ICSE), pages 3–12, 2013. [BIBTEX]
 
[6] M. Tivoli and P. Inverardi. Failure-free coordinators synthesis for component-based architectures. Sci. Comput. Program., 71(3), 2008. [BIBTEX]
 
[7] Marco Autili, Paola Inverardi, Romina Spalazzese, Massimo Tivoli, Filippo Mignosi: Automated Synthesis of Application-layer Connectors from Automata-based Specifications. [BIBTEX]
 
Elsevier Journal of Computer and System Sciences (To Appear).
 

Demonstrators and practical guidelines

This research theme has the twofold objective of continuously experimenting the research outcome to validate and guide the performed research, and to steam out of it practical guidelines for companies and organizations willing to adopt EXOSOUL.
We plan to exercise in the automotive and mobile domains from the very beginning of the project. This will ensure that EXOSOUL builds on real characteristics of these highly-evolving domains and can deliver practical results to implement proof-of-concept demonstrators.  The two scenarios will serve as testbeds and benchmarks for the solutions developed, resulting in rapid feedback for steering the research activities in EXOSOUL.
Concerning  the  automotive  domain,  we  will  exploit  our  collaboration  in  liaison  with  FCA  (Fiat Chrysler Automobiles).  For what concerns the mobile domain, we will mainly experiment in the Android ecosystem.