Three months into this industrial doctorate, I find myself in an unusual position. I was hired as a designer - someone with a background in service design, interaction design, and design research - to work on a project that involves machine learning, federated learning, and data science for vocational rehabilitation. The job description asked for "good understanding of federated learning" alongside design expertise.
I've spent those three months reading: about federated learning as a technology, about the JANUS Pathway Generator and its data archaeology, about models of work rehabilitation from Swedish and international literature, about social networks and their role in occupational rehabilitation. I've also been returning to Krippendorff, whose trajectory of artificiality has reshaped how I think about what "design" means at all in a context like this. The question that keeps surfacing, underneath all of this reading, is deceptively simple: what, exactly, can design contribute to machine learning?
This is not an abstract question for me. It's practical and immediate. I'm a designer embedded in a research consortium exploring ML for public services. What am I supposed to do?
The question is also not mine alone. There is a growing literature at the intersection of design and ML, and a growing professional discourse about what designers bring to AI-intensive projects. But much of that literature assumes conditions that don't hold in my situation - and I suspect don't hold in many situations where ML is being explored in the public sector. This post is an attempt to work through what the literature says, where it breaks down, and what that means for the work I'm trying to do at SCÖ.
The Terms We're Working With
Before surveying the design-ML literature, I need to say something about the terms themselves. "Machine learning" and "artificial intelligence" are used interchangeably in public discourse, and this imprecision matters.
Machine learning, at its core, is a set of statistical and computational techniques for identifying patterns in data. A model is trained on historical data to make predictions or classifications about new data. The "learning" is mathematical optimisation - adjusting parameters to minimise error on a training set. There is no understanding, no judgement, no wisdom. There is pattern-fitting.
"Artificial intelligence" is a much broader and more elastic term. It carries connotations of autonomy, cognition, and capability that the underlying technology often doesn't warrant. When a project manager says "AI", when a politician says "AI", when a funder says "AI" - they may each be imagining something quite different from what a data scientist means by "ML". The term functions less as a technical description than as a vessel for projections: hopes about efficiency, fears about replacement, fantasies about transformation.
Tran Luciani and Lindvall observe that "the current hype around machine learning has probably fueled the misconception of what machine learning is and what it is capable of" (Tran Luciani & Lindvall, 2018). This misconception isn't just a public communication problem. It shapes what projects get funded, what expectations are set, and what designers are asked to contribute to. When "AI" carries the weight of transformative promise, the design brief becomes: help us realise that promise. When "ML" is understood as pattern-fitting on data, the design brief is more modest: help us understand what data we have, what patterns might be useful, and how people should interact with the results.
At SCÖ, I'm encountering the former framing far more than the latter.
This is not just a communication problem. As I argued in my post on Krippendorff, artefacts - including conceptual artefacts like "AI" and "federated learning" - survive or fail in language. The terms carry different meanings for different stakeholders: for the UK researchers, "federated learning" names a specific technical architecture; for the Swedish coordination association, it signals a modern, data-driven approach to a problem they've struggled with; for the funder, it represents innovation. These aren't misunderstandings to be corrected with a glossary. They're different meanings held by different actors within a stakeholder network, and they shape what each party thinks the project is for. Krippendorff would call this a problem of second-order understanding - understanding how others understand these terms - and it's a fundamentally different kind of design problem than building a better interface.
What the Literature Says
A growing body of work addresses the relationship between design and machine learning. I want to survey three main positions before questioning what they assume.
ML as Design Material
The most prominent framing in recent design research treats machine learning as a design material - something designers work with, much as they work with wood, pixels, or service touchpoints.
Tran Luciani and Lindvall argue that "machine learning can be viewed as a design material that is arguably more unpredictable, emergent, and 'alive' than traditional ones" (Tran Luciani & Lindvall, 2018). Where conventional materials have relatively stable properties, ML models change based on what data they see, how they're trained, and how they're deployed. The "material" is dynamic in a way that glass, steel, or even code are not.
This framing opens productive questions for designers: what are the properties of this material? How does it behave? What can you shape with it, and where does it resist shaping? Lindvall's doctoral work on designing with ML in digital pathology demonstrates this approach in practice, showing how designers can develop "designerly abstractions" of ML that enable productive engagement without requiring deep technical expertise (Lindvall, 2021).
Sangüesa and Guersenzvaig push the material metaphor further, arguing that AI introduces new agencies - the results of design processes can now act with a degree of autonomy, making decisions and taking actions in ways that traditional designed artefacts do not (Sangüesa & Guersenzvaig, 2020). A door handle doesn't make decisions. A recommendation system does. This raises questions about responsibility, accountability, and the limits of design control that don't arise with inert materials.
UX Contributions to AI
A more pragmatic strand of literature, exemplified by Lew and Schumacher's AI and UX (2020), focuses on what user experience methods can bring to AI product development. Their argument is straightforward: AI systems are ultimately used by people, and UX professionals understand people. The contribution is in needs assessment, usability testing, interaction design, and ensuring that AI outputs are interpretable and useful.
This represents design at what Buchanan would call the second and third orders - designing products and interactions. It assumes an AI system already exists (or is being built) and asks: how should people interact with it? How should it present its outputs? How should errors be handled? What should the interface look like?
Design for Human-Centred AI
A third strand, which I'll call the human-centred AI movement, asks broader questions about how AI systems should be designed to serve human values. This includes work on explainability (making model decisions interpretable), fairness (addressing bias in training data and predictions), and accountability (ensuring someone is responsible when things go wrong).
This work is valuable and important. But it still typically assumes that an AI system exists, or is technically feasible, and asks how to make it more humane. The design contribution is ameliorative - improving a technology that's already on its way to existing.
Where These Framings Break Down
All three framings share an assumption that troubles me, at least in the current context I am working in: they assume the ML system exists, or can exist, as a material to be designed with.
The "ML as design material" approach assumes you have a material to work with - a model you can train, test, and iterate on. The UX approach assumes a system you can put in front of users and refine. The human-centred AI approach assumes a technology whose harms need mitigating.
At SCÖ, none of these assumptions hold.
There is no ML system. There is no trained model. There is no dataset in a form that could train a model. As I documented in my analysis of the JANUS Pathway Generator, even understanding what data the existing system uses requires significant archaeological work - reverse-engineering conceptual models from Python code, disambiguating Icelandic variable names, identifying which assessment protocols generated which data points. And that's the system that already exists in another country. In Sweden, we're starting from further back.
My concept modelling work has shown that the models of work rehabilitation in Swedish and international literature are diverse, overlapping, and contested. There is no agreed conceptual framework for what "successful rehabilitation" means, let alone standardised data for measuring it. The BIP assessment tool being implemented at SCÖ captures five broad categories; the JANUS system uses over a hundred variables drawn from multiple standardised instruments. These are not the same thing.
The question "what can design contribute to ML?" assumes ML is on the table. What if the more honest question is: what needs to be in place before ML becomes possible, and can design help establish those conditions?
From Products to Discourse
To think about where the design-ML literature goes wrong, I need two frameworks that have been shaping my thinking. One is Richard Buchanan's four orders of design (Buchanan, 1992, 2001). The other is Krippendorff's trajectory of artificiality, which I've been working through separately.
Buchanan argues that design operates across four expanding domains: symbols, artefacts, activities and services, and systems and environments. Each addresses different kinds of problems. Most of the design-ML literature operates at the second and third orders - designing products that incorporate ML, or designing interactions with ML systems.
Krippendorff's trajectory maps a parallel but, I think, more revealing expansion. Design has moved through five classes of increasingly complex "material": products, goods and identities, interfaces, multi-user networks, and finally discourses - "living in communities of people who collaborate in their production" (Krippendorff, 2011). At each level, the material designers work with becomes less tangible and more social. At the product level, you shape metal and plastic. At the discourse level, you shape meanings, narratives, and shared understandings.
What makes Krippendorff's framework particularly useful here is his distinction between first-order and second-order understanding. First-order understanding is the engineer's understanding - how things work according to specifications. It's mono-logic: one rationality that everyone should comprehend. Second-order understanding is understanding of understanding - recognising that different stakeholders bring different rationalities to bear on the same artefact, and that these differences aren't errors to be corrected but the actual terrain designers must navigate.
The design-ML literature is overwhelmingly first-order. It treats ML as a technical material with properties to be understood and shaped - a mono-logic framing. The designer learns about neural networks, training data, and model behaviour, then applies design methods to this technical material. This works when the ML system exists and the design problem is about interaction, interface, or user experience.
But at SCÖ, I'm not working with a technical material. I'm working in a discourse - a space where "data science", "federated learning", and "AI" carry different meanings for different stakeholders, where the very definition of what we're trying to do is unstable and contested. The design problem here isn't shaping a material. It's navigating a multi-logic landscape where researchers, practitioners, managers, and policymakers each understand the project through fundamentally different frames.
Krippendorff argues that at the discourse level, design must embrace "chaos, heterarchy, diversity, and dialogue" - the opposite of the optimisation logic that underpins ML. This isn't a weakness of the situation. It's the nature of the material. And it means the question "what can design contribute to ML?" is pitched at the wrong level of the trajectory. It assumes we're working with products or interfaces. We're working with discourse.
What Design Might Actually Contribute
If I reframe the question from "what can design contribute to ML?" to "what can design contribute to understanding whether ML is possible here, and what would be needed?", the design contribution looks quite different from what the literature typically describes.
Conceptual Modelling
My concept modelling work represents one kind of design contribution. By reviewing models of work rehabilitation from multiple sources - Swedish assessment frameworks, the ICF classification, the JANUS variables, the BIP instrument - and visualising them as conceptual models, I'm trying to make visible the heterogeneity of how this domain is understood.
This is design at the level of representation and sense-making. It doesn't produce an ML system. It produces shared understanding (or at least, surfaces the lack of shared understanding) about what we would be modelling if we built one. It reveals that different stakeholders and different national contexts conceptualise "rehabilitation success" in incommensurable ways - and that any data science approach would need to reckon with this conceptual diversity rather than assuming it away.
Data Archaeology
The work I've done on the Pathway Generator patient vector is another form of design contribution. Reverse-engineering conceptual models from code, disambiguating variables, identifying assessment protocols - this is a kind of information architecture work. It makes legible what was previously opaque.
It also exposes uncomfortable questions. The JANUS system uses over a hundred variables, many derived from standardised instruments (DASS for depression and anxiety, Rosenberg for self-efficacy, SIAS for social anxiety). Do equivalent instruments and data exist in Sweden? Are the Swedish organisations gathering comparable data? If not, what would it take to start?
These are precondition questions, not product questions. Design's contribution here is diagnostic: understanding what exists, what's missing, and what the gap means for the project's stated ambitions.
Surfacing Tensions
Design can also contribute by making visible the tensions that the "AI for public services" framing tends to obscure.
My post on social networks in rehabilitation highlighted one such tension: between computational approaches (which emphasise patterns, prediction, and efficiency) and relational approaches (which emphasise trust, connection, and human judgement). Wastell's concept of "technomagic" - the tendency to see technology as a magical solution to complex social problems - names the risk that ML promises transformation without reckoning with the conditions that make transformation possible (Wastell, 2011).
Design can surface these tensions not by resolving them but by representing them in ways that force honest conversation. This is closer to what Schön called "problem-setting" than "problem-solving" - the work of framing what the problem actually is, before rushing to solutions (Schön, 1983).
Discourse Work
Perhaps the most important design contribution - and the hardest to articulate or justify - is what Krippendorff would call discourse-level work. At SCÖ, the words "data science" and "federated learning" function as what one can call discursive artefacts, or 'floating signifiers'. They circulate through the project, shaping expectations, attracting funding, aligning (or misaligning) stakeholders. Different people invest them with different meanings. For some, "federated learning" means a specific cryptographic protocol; for others, it means "we can share data without sharing data"; for others still, it means "innovation".
The design contribution here is second-order: not building the technology, but understanding how different stakeholders understand what the technology is and could be, and creating conditions where those understandings can be surfaced, compared, and - where possible - aligned. This might involve conceptual models, workshops, prototypes, or simply careful conversations. But the "material" being shaped isn't code or pixels. It's meaning.
Krippendorff argues that designers who work at the discourse level must "leave artefacts underspecified, open, enabling interested others to be designers of their own worlds, in their own terms" (Krippendorff, 2011). This is a radically different posture from the design-ML literature, which typically positions the designer as someone who shapes a product for users. At the discourse level, the designer's role is more like facilitation - creating the conditions for stakeholders to shape their own understanding of what's possible and desirable.
The Question Behind the Question
There is a version of "what can design contribute to ML?" that I think is the wrong question. It assumes ML is the destination and asks how design can help us get there. It positions design as a support function for a technology project - smoothing the path, improving the interface, making the outputs more humane.
The version I'm more interested in is: what can design contribute to understanding whether ML is the right approach, and if so, what kind of ML, for what purposes, with what data, in what institutional context?
This is a harder question, and it may produce less comfortable answers. Design's contribution might be to reveal that the preconditions for ML don't exist. That the data isn't there, the governance isn't ready, the conceptual foundations are contested. That the technology imaginary - federated learning as a solution to data silos - assumes infrastructure and alignment that would take years to build.
That's not a popular contribution. Nobody funds a design researcher to conclude that the technology the project is built around isn't feasible. But it might be an honest one.
Herbert Simon famously defined design as the transformation of existing conditions into preferred ones (Simon, 1969). But as Krippendorff argues, Simon's framing celebrates hierarchy and mono-logical rationality - a single designer or design team determining what's "preferred" and engineering the transformation. In a domain like vocational rehabilitation, where multiple stakeholders hold legitimate but incompatible understandings of what "good outcomes" look like, Simon's framework is insufficient. The question isn't just who gets to determine which conditions are "preferred", but whether any single rationality can determine this - and whether a technological intervention is the right means of transformation when what matters is relationships, trust, individual circumstance, and the slow work of rebuilding capacity. The preferred condition might not involve ML at all. Or it might, but only after a great deal of groundwork that looks nothing like building an algorithm.
I don't know the answer yet. I'm three months in. The concept modelling work continues. The data landscape is becoming clearer but also more complex. The gap between what the project describes and what the institutional context supports is becoming more visible.
What I'm increasingly sure of is that the question "what can design contribute to ML?" needs to be preceded by another: what is actually going on here, and what does this situation actually need?
References
Buchanan, R. (1992). Wicked Problems in Design Thinking. Design Issues, 8(2), 5-21.
Buchanan, R. (2001). Design Research and the New Learning. Design Issues, 17(4), 3-23.
Hill, D. (2022). Designing Missions: Mission-Oriented Innovation in Sweden. Vinnova.
Krippendorff, K. (2005). The Semantic Turn: A New Foundation for Design. Taylor & Francis/CRC Press.
Krippendorff, K. (2011). Principles of Design and a Trajectory of Artificiality. Journal of Product Innovation Management, 28, 411-418.
Lew, G., Jr., R., & Schumacher, R.M. (2020). AI and UX: Why Artificial Intelligence Needs User Experience. Apress.
Lindvall, M. (2021). Designing With Machine Learning in Digital Pathology [Doctoral dissertation]. Linköping University.
Murray-Rust, D., Nicenboim, I., & Lockton, D. (2022). Metaphors for designers working with AI. Proceedings of DRS2022. https://doi.org/10.21606/drs.2022.667
Sangüesa, R. & Guersenzvaig, A. (2020). AI as a Design Material: Dealing with New Agencies. Temes de Disseny, 35, 6-25.
Schön, D.A. (1983). The Reflective Practitioner: How Professionals Think in Action. Basic Books.
Simon, H.A. (1969). The Sciences of the Artificial. MIT Press.
Tran Luciani, D. & Lindvall, M. (2018). Machine learning as a design material: A curated collection of exemplars for visual interaction. Proceedings of NordiCHI 2018.
Wastell, D. (2011). Managers as Designers in the Public Services: Beyond Technomagic. Triarchy Press.
Yang, Q., Scuito, A., Zimmerman, J., Forlizzi, J., & Steinfeld, A. (2018). Investigating How Experienced UX Designers Effectively Work with Machine Learning. Proceedings of DIS 2018, 585-596.