Talk Summary

In this Panel discussion members of the DataEthics4All Leadership team discuss Corporate Ethics and the Role of Whistleblowers in organizations in light of the recent WSJ leak from former Facebook product manager Frances Haugen on the harms it has caused and ignored.

0:01 Sam Wigglesworth:

Welcome to our first panel of the day, my first panel and the team’s first panel on corporate ethics; we’re going to be looking at the role of whistleblowers. I’m going to be moderating, I’m Sam and we have our panel which is our amazing leadership team; Kevin Dias is here, as is Bruce Hoffman and Susanna Raj. Really looking forward to it! The main discussion point today is the recent news about Frances Haugen’s report and testimony to Congress about her experience at Facebook and the interviews that she’s had with media outlets, and what our thoughts are on on the points that she raised. So, as a starter question to the panel: when just a few days ago we couldn’t access our Whatsapp chat, we couldn’t get on Instagram, we couldn’t update our Facebook profiles – we had a few hours out of social media; how did you feel when that happened?

1:30 Bruce Hoffman:

It’s kind of funny because in my case I would say it’s probably good for everyone to have a rest from social media, any possible time that can actually happen, but at the same time in thinking about that and listening to what’s been said, you realize it’s not just a rest from social media but it’s a break from people being able to do business; a break from people being able to communicate, because of all the dependencies on the platforms that Facebook owns. I think it was 2.8 billion people that are actually communicating across the multitude of platforms that went down. So that being down just made it feel like there’s too much all in one place and it’s almost a risk to world communications to have that be at a company level rather than some ownership beyond it, so it’s an interesting challenge that that we’ve put ourselves in, and then of course hearing about it, it almost feels kind of crazy that it’s that big and that much impact could really occur. Personally I didn’t even realize it was going on until somebody on the panel shared it with me, because I wasn’t on at that point!

2:54 Sam Wigglesworth:

Yeah, that’s a great point – we rely on it so much, it’s a significant number of people, it’s global and it affects us all. What are your thoughts then Kevin or Susanna? Are you happy with the fact that you didn’t necessarily have to access social media for a few hours or do you rely on it, for business for example?

3:17 Susanna Raj:

I don’t rely on it for business, but I realized how many of my family and friends that are from my native land (India) are using Whatsapp to connect with me, not my traditional phone number because that has an international cost, so they were all using Whatsapp and then to see that they’re not able to reach me and I was not able to reach them – I just realized how much I was relying on this free service without thinking that that could be hacked away from me! Maybe I am the product now – whenever something is free they say you are the product they are selling, so I just realized: okay, now I need to have a backup plan, something that is paid so that I know I can access it. And also to realize how insecure the whole process behind the scenes was when I was reading the stories; even though I could not understand the technical terminology used I still could see that it was so incompetent; that really shocked me. When they talk about end-to-end encryptions you feel so secure, but the security level is as pathetic as me using the same password on different accounts; they were doing the same thing at a different level. So that also made me feel less trusting of those mediums that I’m now using, especially Whatsapp. I did not even know Instagram or Facebook were down because I’m not even on those platforms, but I realized Whatsapp was down, and that’s what made me go and look at the news to see what was going on.

5:16 Kevin Dias:

I think if I could add to what was already said: we’ve put ourselves in this place where we’re so reliant on Whatsapp. I think it’s funny, Bruce’s dog (barking in the background) echoes all the sentiment of my extended family also back in India on that day for those five hours; I think everyone there uses Whatsapp, some of them use Facebook, not too many use Instagram. When this happened they had no way of reaching out to me, and we reach out to each other every day and there was a huge panic; they were like ‘what’s going on, is the internet down?’ That was a funny comment that my mom made – ‘is the internet down’, because that’s all they do on the internet, it’s just Whatsapp and Facebook. I remember a few months ago, I had actually tried to get them on other instant messaging apps like Telegram, and another comment now comes to mind that my mom and dad had made then: ‘do you really need another app, isn’t Whatsapp enough? It’s amazing, it’s so easy to use’ – all those kind of things, and now that I think about that, I’m realizing that Facebook has made it so attractive from a user experience perspective. Not a lot of people are willing to make the leap to change to a different app of any sort, and we all know that there are different instant messaging apps that are out there but most people prefer Facebook and Whatsapp.

7:01 Sam Wigglesworth:

Yeah, definitely. We do rely on it as chat tool and also for sharing images and audio, particularly in a country like India where there’s almost a dependence on it. There are alternatives out there too, but sometimes we rely on one application, one platform that potentially puts us at risk of not being able to communicate with our family and friends, if you’re able to just turn the power off and then globally not talk to close family members. Definitely worth thinking about for the future. The other thing we were discussing as well, that Frances Haugen brought up, was safeguarding and the importance of making sure, particularly with Instagram, that we look after our privacy rights and consider the harm of platforms like Instagram on young people. Do you think that there’s an issue at the moment with platforms like Facebook and Instagram, and is it having an impact on young people, particularly young girls and their health, or do you think that it’s okay for them to use it as frequently sometimes as they do; what’s your view?

8:29 Bruce Hoffman:

I think it’s pretty clear that everything that we hear, read, see and study shows that there’s an impact; things have started. I mean it was a lot more Facebook originally with shaming and people committing suicide as a result; there’s been quite a few studies on the dependency. It’s kind of interesting, going back to the previous piece of the conversation, if we go back in time we didn’t have this dependency; if you couldn’t communicate for five hours, that wouldn’t have been a surprise. You might have said: ‘hey, I could communicate within a day’; it’s that dependency on things being instant that has changed the entire generation, and you’re right, everybody’s expecting instant feedback. If you don’t get positive feedback and you’ve created this dependency that’s really occurred on social media across the board, people begin to feel real pain – it’s psychological; there’s mental health issues that have cropped up and are getting significantly worse, and it’s just odd that society has created such a dependence on this. We’ll talk tomorrow a little bit about Covid, but it’s only increased our dependency on this connection. I like what Kevin said a moment ago – ‘is the internet down’, right? It’s like the internet is now water to us – it’s the same as air, electricity, food; the internet is the next piece of that, we can’t do without it, and it can cause a lot of pain from people just thumbs downing or creating negativity around what we post. I’ll let others speak about it, but I know it’s a pretty common theme that we hear.

10:26 Susanna Raj:

I think what struck me from Frances Haugen’s interview and also from reading her brief on the documents that she released to the public is that Facebook was putting profits over the public good, what was good for the company was not good for the public, so it was not even aligned properly. Building on conflict is their entire business model, so there was no way for them to get around it; even though they had a civic integrity panel put in before the elections. They removed the panel once the elections were over, and were like ‘okay, the election went fine, let’s dismantle this panel now, we are okay’ – and then January 6th happened and they were like ‘oh no, it was not okay’. But they just didn’t realize – what they were doing was optimizing for engagement; by naturally optimizing for engagement they were optimizing for hatred and optimizing for negativity and also for anti-democratic forces. What are your thoughts on having a business model that is diametrically opposed to the public good?

12:27 Shilpi Agarwal:

I know I’m not on the panel, but I think this is such an interesting question! I think that how the corporate board is designed is why they have incentivized profit; to the point that even if you want to put ethics at the forefront, sometimes it is hard. I don’t justify whatever is happening with Facebook, and now they are a very big company and they can do whatever they want, but imagine something similar happening to a smaller startup. That starts with the design and the senior leadership; I always say that ethics has to start from the top down, it can’t go from the bottom up. But if the company doesn’t make profit, it’s a cycle – you don’t get to the second round of funding or the third round of funding unless you show that you’re making profits. Not necessarily that you have to incentivize profits over the user, but this whole funding mechanism works that way. All I’m saying is that you have to put in a conscious effort to make that decision, draw that line, and be ready to walk away from the next round of funding to stick to your ethical principles. It’s not going to be easy but somebody has to do it.

14:14 Bruce Hoffman:

The only thing I was going to say in there was on sensationalism – if we go back to the comments I was making to the team, if you were to watch the news, 95% of it is sensationalism and showing all the negativity. What started before this is only continuing; and Susanna you made a really good point to me, that it has only become worse because now people are interacting with it directly where before they really couldn’t do that with the news. It’s almost like Facebook and any others just took what the world was really asking for, and just made it worse. Just saying: ‘you really now can access, instantly, all this information, now react to it and put your own spin to it.’ You can have viral stuff that’s fun, but boy, you can have viral stuff that just has incredible negativity. It seems like it should be their responsibility to do something about it. I think that gets back to something said at the beginning about whether regulation is coming, and will it help? Or is it always just going to be too late and not enough?

15:34 Shilpi Agarwal:

Regulation is a very low bar. To your point, has someone seen the Apple News series on the morning show? It’s exactly what Bruce talked about – all the media people always say they only tell what the audience wants to hear, they only make films that the audience wants to see, they only bring out the role models in society that the society believes in or wants to see at that time. It’s true and not so true, because if media people don’t do their job with bringing things that people are not ready for, how will people ever be ready for the right thing?

16:26 Susanna Raj:

It was really telling that in response to all this backlash, the policy director of communications at Facebook; this was her official response, she actually said: “solutions have not yet been found by anyone, so if you know of any company that has found a solution to us, let us know.” I mean, seriously?! What do you guys think, have solutions not been found yet to this problem?

17:05 Bruce Hoffman:

It sure seems like there are!

17:06 Kevin Diaz:

Companies just need to think about it a little bit more, rather than just saying that there won’t be a solution or that it’s impossible, that kind of thing. To Shilpi’s point as well, ethics starts from the top down, I feel like in the future once C-suite level executives start appreciating people more than their profit margins – easier said than done, because ethics is opposite to their business goals – that’s the direction we need to head towards. Looking at people as people rather than just numbers. Again, it’s easier said than done, but if we have that mindset I think we can come up with solutions towards this.

18:02 Sam Wigglesworth:

I think you’re right, it’s a really good point. I think it’s about making C-suite executives a little bit more accountable, building a bit more transparency within organizations and knowing what’s happening if you’re building a piece of technology that could potentially change the world and benefit people. It’s about putting the user first. We need to know what’s happening, how our data is being used, how the algorithm works within a particular application that’s been developed, and truly understand, to build that transparency and trust. We need that guidance as well, companies need guidance and those frameworks as well to support them.

18:56 Bruce Hoffman:

We keep hearing about all these new services that are are also free that actually protect your privacy, but the other things have become so big, how how do you even really make a change? It almost feels like it’s such a small place – I mean, you’re right, I think we can all agree that change is needed. That’s an easy thing to say; making it happen is difficult. What we’re here for is to try to help; all of our voices I think have some some good agreement there, but all we can do is continue to push.

19:37 Sam Wigglesworth:

I think Frances Haugen also said that social media can be used for good, and we need to focus on that – positive AI and social media for the future. How we can create that trust is also important, carrying on with that message as well. It can be used for good and we don’t want to turn away from it. It connects billions of people across the world and we rely on it.

20:17 Susanna Raj:

So that brings us to the question of moderation: how to moderate content, how to moderate opinions that you don’t necessarily disagree with. While somebody is behind the scenes, moderating the content of Facebook, they have to look at the freedom of speech of both parties and see which ones should they moderate and which one becomes hate speech – where do you draw the line? Those are the kind of questions that we need to ask ourselves also, when we post content and comments – it’s our freedom of speech, but then how far should it go, where are the boundaries? What does everyone think on that?

21:00 Kevin Diaz:

I remember studying this and I know that the conclusion of some of the research that I’ve read is that the human brain is just naturally attracted to divisiveness. I think Facebook realized this early on, and obviously it’s not the best thing to leverage or take advantage of, but they did. That’s why when you see the comment sections it’s usually more than just comments, they’re arguments, most of the time. I think another realization that Facebook had early on is that a lot of engagement with or consumption of the content itself happens in the comment section. A lot of people prefer finding their information from the comment section versus the actual content that was posted, just to maybe validate what was posted or find out more information, get reviews and opinions on it. All of that combined, I think Facebook is just leveraging people’s emotions that way, the brain’s natural inclination towards divisiveness.

22:13 Susanna Raj:

We are socially motivated – if you share something like a news article with me, and then you post a one- or two-liner, I’m more interested in what you said than content itself. When we are socially connected with each other we trust the person who has sent that content to us more than the content itself, and we don’t bother reading it and seeing whether it actually agrees with our opinions. I won’t say that Facebook did not know the comments section would be their most engaging, profit-making section – they do know that, and that’s why most of their algorithm works behind the scenes on the comment section, not just on the post. They do know; they did have psychologists there to manipulate us, but I wish they would have psychologists on board to manipulate us in the other direction as well.

23:19 Sam Wigglesworth:

It’s a good point, we don’t want to be dividing and going against free speech or democracy, but I think the amplification of negativity is something that needs to be looked at. We need to continue having that debate on how we move forward with that to get the balance right.

23:45 Bruce Hoffman:

At least amplify positivity also.

23:48 Sam Wigglesworth:

Yeah definitely. More transparency definitely, a more positive future and starting from top down as Shilpi said. Do you have any other comments to make about what was raised? Frances covered quite a lot but do you feel that, for example, we can move forward and still use these tools for good and do you think we’re going to be moving forward towards greater data privacy and protection? What are your views on the future in terms of social media and some of the risks?

24:45 Kevin Diaz:

I wanted to also address, because I know we talked about it, mental health – rather we mentioned it, but we didn’t unpack it enough, we spoke about it particularly in the context of the younger generation. Besides the whole community aspect of Facebook, there’s also these features and tools within it that we should also acknowledge, such as the use of filters. From a content creation standpoint, I think when we talk about mental health, filters also need moderation; I know we talked about moderation of comments and discussions, but filters as well need moderation, or curating at least, because as we know our filters create this superficial sense of beauty or attractiveness, and that spirals into most of the mental health challenges that we face these days. Thinking about things like these specific tools and features as well is important. I know that the community aspect, the instant messaging aspect is also important but I think these features will become more relevant in the future as well. To the point that was addressed earlier about taking all of these negative challenges and finding the good in them, filters have been used for good – for example virtual shopping. Maybe explore that a little bit more, versus the way it has been used negatively or in a more addictive, toxic angle so far.

26:44 Susanna Raj:

I think we do have some questions as well, right – so maybe we should take those?

26:51 Sam Wigglesworth:

Yeah, let’s do that. Alberto says: ‘if this is an internal effort, should we not be bringing third parties on board to help find solutions?’ What do we think?

27:14 Susanna Raj:

That’s always the question of internal versus external audits. I don’t think it’s gonna work unless it’s internal. In countries like the United States we have traffic lights and traffic signals, and everyone follows it even in the middle of the night when there are no cars on the road; you will instinctively stop at the red light even though there is nobody there – that has to come from inside; so it is it has to be internal first and external next. That is my opinion: external rules can be there but people might not comply with it, everybody from the top to the bottom has to internally feel it, and it has to be applied in that realm where you will do the right thing even if nobody’s watching you.

28:06 Bruce Hoffman:

That’s right, it’s a cultural change that we would hope happens across the board, although we know that when we look at the world there’s so much hate and challenge, that it’s hard to see the cultural change actually occur. Who decides? I think the people decide to actually do things differently – it can’t be forced, and the corporations tend to just take advantage of whatever they can until they’re really pushed to do something different.

28:41 Shilpi Agarwal:

Yeah, and I think that’s why organizations like DataEthics4All have an important role to play. What do you guys think about that?

28:52 Kevin Diaz:

Yeah a third party could be us or an organization like us, but I think the comment also asks whether this third-party resource should be brought on board to help find a solution, but I think the question really is: even if there are solutions, will the company implement it? Because for them it’s more of a question of the profit of that solution. With solutions also comes testing, and just that experiment phase of it can cost the business millions of dollars, depending on the business – if we’re talking about Facebook still – and I think Facebook or companies like Facebook would just be hesitant to put in that work. But still, there’s definitely good that can come out of suggesting these solutions, and hopefully Facebook does take that leap of faith and try out some of these good solutions that are out there.

29:50 Shilpi Agarwal:

It’s all about cultural things like those we touched upon. We have to draw a fine line; we have to walk this very difficult line where we have to make an ethical choice. It’s like everything in life, right? My parents always said, I’m sure everybody’s parents said the same thing, that whenever life gives you a fork in the road, you can take the easy path that seems to be shorter and easier right now, that is not doing the right thing but will lead you to greater success in the short run. But the longer path – the righteous path – may take you longer to get there, but it is the right way to get there. I think on that note, thank you my panelists and my leadership team, thank you so much for this wonderful inspiring discussion! I’m sure the discussion will continue – we can’t solve this in one discussion, but it’s the start and very much needed.

Talk Summary

In this Panel discussion Susanna Raj [DataEthics4All Leadership Council]; Professor Michael Witbrock, University of Auckland and Raluca Crisan, CTO & Co-Founder, Etiq AI explore how realistic the mitigation strategies are today when it comes to mitigating Bias in AI.

0:09 Shilpi Agarwal:

Next up, we have a panel discussion that should be a fiery one. Bias in AI is what everybody’s talking about, but how realistic are the mitigation strategies of today? We have a very diverse perspective from the panelists. We have Professor Michael Witbrock from the University of Auckland and Raluca Crisan who is the CTO and co-founder of Etiq AI. Moderated by our very own Leadership Council, Susanna Raj.

1:07 Susanna Raj:

Welcome everyone, thank you for joining me on this panel. Today we are going to discuss bias mitigation strategies that are being used today and how effective they really are. I’m excited to have an academic perspective on this as well as a business perspective from Raluca, the co-founder, and CTO of Etiq AI, and Michael from the University of Auckland. I just want to start this with a very basic question. Because bias means different things to different people who come from different backgrounds, it means it has a totally different meaning for me as a social science researcher – so I want to start with a basic question: what does bias mean to you? 

1:59 Michael Witbrock:

I think it means at least two completely different things. And I think this makes it difficult when it’s used. It can just mean the predisposition in data in some systemic or particular way – so the data suggests some sort of action or suggests some categories that cause the system to behave in a particular way, and that that’s built into the way the system is produced. Or it can mean the sort of invidious predispositions that some human beings have which cause them to behave in ways that we would think are undesirable or even evil. These two things are not exactly the same but they are used interchangeably, and not to our benefit.

2:56 Raluca Crisan:

Within the context of my work and the ethical field, bias is one of those things that is easy to recognize when you see but a little bit hard to define in a standardized way. It’s basically an outcome of an automated decision system which is somehow obviously unfair when you think about it. It could be that an overall system performs much worse from an accuracy point of view for a certain group; so it just doesn’t seem to work for that group. That’s something that if we see we say, ‘oh yeah, that’s a problem, that’s unfair’. Or it could be that we have two individuals who are very similar in all aspects we consider important for getting a loan or getting a job, but maybe they’re different genders – something we don’t consider important for that as humans. And then the system comes up with a biased decision – something we recognize as unfair. It’s hard to define but we can recognize it.

4:11 Susanna Raj:

And I do think we are all born with our own biases. We have evolved with our own biases, some of them have protected us, some of them I think we need to get rid of soon – but we are a work in progress, right? But the biases are embedded in our data as well.

4:27 Michael Witbrock:

One of the things which are so very interesting – the source of this debate – is that as we build systems with the data that human beings have produced, they make those biases. Whether they’re good or bad, biases manifest. They allow us to experiment on those biases.  I have a little bit of a problem with this use of the word ‘algorithmic bias’ – it’s not that the algorithms have biases. Maybe the people applying the algorithms have biases… alright, it’s at least logically possible that the algorithms have biases based on the way they develop, but that is almost never what we’re talking about. The data that’s been selected is representative of a human group, and possibly all of the humans over some time period. The decisions that these systems make reflect those biases and make it clear to us, for example, that in our literature we are teaching ourselves to have biases that might well be expected to cause humans to discriminate against women, for example. And by having computers do this, we can point the finger at them and say “bad computer!”, knowing, I hope full well, that it means that we’re bad people too.

6:00 Susanna Raj:

I do agree. There’s this interesting debate going on whether the algorithmic bias starts in the algorithm or it starts in the people or it starts in the data. I feel like that’s a chicken and egg discussion that we will continuously have forever. So what do you think Raluca, on that note?

6:24 Raluca Crisan:

For sure, there’s this approach or mindset that it is about the data. And as I said, for sure it’s definitely a big part of the way things get decided and it’s reflecting our biases and so forth. But it can happen across a variety of things; so for instance, as humans, we make some decisions that are biased based on characteristics that are biased; but as an algorithm, I might just be picking up a proxy. So it’s not something that maybe as a human I would ever recognize to be associated with a certain demographic group, but maybe the algorithm somehow recognizes it and part of the algorithm using that feature is that maybe it has some real explanatory power beyond the bias, but part of it is the bias, and then which one is which is a very tricky problem to assess. So yes, it’s the data, for sure, but it’s also maybe the nature of the system that is discovering patterns in the data, and then how we interact with that system and how we try to shape it and control it. If we don’t want it to do certain things, we can affect that.

7:55 Michael Witbrock:

I think the fundamental difference there – when you say that a human being would never consider doing this, you never want to speak too soon on these things, human beings can be quite awful! But to the extent that human beings wouldn’t manifest the negative biases in the data that they’re learning from – to the extent that’s true, it’s because we have access to a kind of data that these systems don’t have; and ironically we have access to a source of inductive bias, which these systems don’t have, and that source is knowledge. Why can we read the same data that GPT-3 reads about suitable jobs for women and men and not reliably produce the same predictions (across many languages) that the engineers will be male and the nurses will be female? The data causes that to happen if you make a naive language. Why do we know better than to exhibit those biases? We know better because we know better; so we have a piece of knowledge that tells us that we should disregard that sort of information in many decisions because it will cause unjust actions. And as we move towards computers having access to that kind of knowledge, I think that the likelihood is that quite soon, we will be less likely to have decisions based on invidious biases from machines than from people. So I think a lot of the problems we’re having at the moment – not all of them but a lot – are very temporary, and they’re because of a weakness in our AI systems, namely the weakness in the inability to use not just data but knowledge.

10:24 Susanna Raj:

I think this follows very well into our next question – if you think it is so complex and so embedded in the data itself, where do you think, as a solutions-finder, while we are working on a solution to solve this problem, where do we start in the development cycle; where do we start in the AI life cycle? Where do we start solving this problem? Are we starting at the wrong place, when we look at only algorithms or mitigation strategies? Or should we actually be starting at the data collection or the idea consumption stage? So where do you think we should start? 

11:09 Raluca Crisan:

Definitely as early as possible, but I think it’s better to start somewhere! So I think that would be my first thing, it’s better to start somewhere we can at least make some sort of an impact. But yeah, it’s definitely early in the life cycle, and there are just a few things we’re noticing from what we’re doing. One, it’s not about the metrics; we’re finding metrics not super helpful. It doesn’t seem to be an optimization problem. I think a lot of the literature is heavily around it as an optimization problem, which is natural because this is the world we all live in, but it doesn’t follow that the solution can necessarily be solved in this way. So what we’re seeing that is a little bit more effective is whether it is at the data or pre-data collection stages or at the output stage. It’s around that kind of interpretability layer, surfacing issues, and then the human can interact with those issues, and in a sense, trace them back. But just to give you an example, for instance, when we say a data collection; let’s say you’re a company that targets certain demographic groups, let’s say white males between 35-65 and a certain income bracket in a certain US state. It’s very hard for you to collect data; your whole product is optimized for this demographic. So when we talk about data collection, it can lead the company to start asking uncomfortable questions about what it is they’re actually doing as part of their business model, because if you’re marketing that your product is for this demographic then you will collect a certain kind of data.

13:09 Michael Witbrock:

I think that one of the things that we’re missing is any sort of clear agreement on what these classes of invidious biases are, from the point of view of people who are building decision support systems. We live in a sort of hodgepodge of ideas from different countries. New Zealand, where I am, differs quite strongly from say, the United States in the extent to which it enforces protections against violence against women or against LGBT people. And certainly, New Zealand differs very strongly compared to some other countries. Suppose we’re trying to build a system that identifies, for example, whether these proxies are being used. You could build a system that looks at your data and sees whether the data has features that can be used as proxies for identifying classes, such as likely vaccination status, or self-described gender. Suppose you build a system that was trying to see whether there are things that are used as proxies for that, it would be very useful to know what sort of things we should build in as required checks for proxies. It turns out, based on a probably to be found out to be bogus study, that your astrological sign predicts your vaccination status in the United States. There’s a news story about that today, I hope we find out that that’s not true! It was 46% versus 70%. If it is true, your posterior probability to astrology would be higher, so one assumes that’s not true. Suppose that was true and you can predict birth month from vaccination status, should we build into systems attempts to detect that, attempts to predict month or gender? Only if they can’t predict that using these sorts of proxies should we allow them to go forward. Only by starting to have a discussion about what the industry standards should be for testing these things irrespective of what the particular countries think will we be able to make progress using the techniques that we’ve got available to us at the moment.

16:54 Susanna Raj:

I think proxies by their existence itself actually stand for societal biases. I mean, all our proxies have a bias built-in, and that bias comes from prejudice; and historical discriminations are built into those proxies. I believe that even using a proxy for crime prevention, or anything else, we have to look at the knowledge behind it. How did we come to know this proxy is a good stand-in for that classifier? So to me that that itself is a dangerous zone to go into, but what do you think Raluca? You are actually in the industry itself, working on solving this problem, so I would like to hear your opinion on using proxies and what Michael was saying so far.

17:50 Raluca Crisan:

Obviously, it’s hard to generalize, but I would say that whether intended or not, it does happen that people use this. But of course, a proxy is never just a proxy, so it’s a very gray area. The problem is when people are not even aware; when they’re not really looking at this topic at all. I’d say that’s a starting point of a problem. If they at least have some understanding that their models might impact actual people in negative ways, at least they can start looking at it and treating it with a certain degree of caution. I think even before that there’s a step that needs to be undertaken. I’m not sure if this answers the question, but it’s very hard to regulate, right? First of all, what’s the threshold for it; how would you define it? There are a few ways to calculate basic metrics, just the basic metrics, so it’s very hard to regulate as such because you wouldn’t be able to define it in a standardized way. But if you look at the data, look at the model, look at the outcomes, you can pick up if something is wrong. And you should be looking at this and picking up if it’s wrong, right? So this is I think the dubious area. I’m not sure if I’m right, but if the people that are working on this are starting to look into it a little bit more then they will at least try to manage within their applied use case to make the decision less problematic, so they will say: ‘actually, income is a proxy for financing, it’s a general decision that I’ll use this. But if I show up with a certain brand at a certain time of day, maybe I shouldn’t use this because actually, it’s a proxy for something that maybe can be a demographic group’, right? These are things that happen, like: the example is clearly wrong, but if you don’t even look for it, you’re never going to find them, you think it’s all good.

20:29 Michael Witbrock:

Well, this case with income versus ethnicity or agenda is an interesting case, because they could show up as proxies in both directions, and the reason they’re a proxy is that they’re actually causally related. Right? People of certain ethnicities or certain genders have lower incomes because of discrimination. This isn’t an accident, so one could argue, for example, that by removing the knowledge of the effect of gender on income from your system, you’re likely to be preserving discrimination. This is actually related to the affirmative action question. You’re likely to be perpetrating discrimination by removing the system’s ability to notice that income is dependent on gender and therefore should not be allowed to affect the outcome for somebody. Of course, people have wildly different opinions about this. Their opinion varies wildly depending on which groups they’re members of or are not members of. Who this generation should be mitigated against also varies widely. So it’s much easier just to blame the algorithms.

22:23 Raluca Crisan:

I’m not sure this is at all helpful, but for instance, in some countries, you can’t use the demographic feature in the model itself. The regulator says ‘no’, for sure. So then people are using a proxy for it, but then the problem is then they get counterfactuals. This is what happens; either you have no counterfactuals, but your model is pretty bad for a group, or your model is actually great and you have loads of counterfactuals just moving around and potentially creating issues. How would someone regulate that and solve for it, it still has some question marks.

23:09 Shilpi Agarwal:

We do have some questions. Rakesh Ranjan says: “Bias in data (at least system of records, if not the system of engagement) may represent the state of bias and prejudice in our own society. So I’m interested in learning the panelists’ perspective on if mitigating bias in data would really solve the root cause?”

23:41 Raluca Crisan:

I believe you can. Not completely remove it, but I think we started at the point at which humans are pretty biased, we put these statistics in place to notice that humans are pretty biased, and now we’re like, ‘Oh, let’s fix this, we’ve made the systems more complicated now; it’s there but now we notice it’s reverting back to ‘oh, let’s just let the humans be biased’ – that’s probably not the way to go. There’s potential for this to make a difference for the good overall, maybe.

24:13 Michael Witbrock:

Yeah, I think it can certainly mitigate the root cause, if nothing else, by allowing us to build systems that can point out the causes and the likely effects of human bias as well as bias in the system from human-derived data.

24:41  Susanna Raj:

I know Michael is very much interested in building systems of knowledge as a way of mitigating bias. I would like you to elaborate a little bit on that because that seems to go back into the foundational models of AI and look at whether we can use knowledge as a whole or, as Raluca pointed out, there are proxies. I also wanted to jump in and say that we actually used another proxy for income: marital status and education level as a proxy, but even that led to problems! So, we cannot even start with proxies anymore, but how do you start with knowledge as a database, instead of looking at data?

25:36 Michael Witbrock:

Knowing that something is a proxy for something else, we can therefore try to build casually informed models which mitigate those biases. That’s a kind of knowledge: what goes into these knowledge structures. We have to be able to use these features because we can’t build mitigation systems without them. If you can’t know that your system will use income as a proxy for gender, for example, it’s impossible to mitigate that effect, because you can’t build a causal model which supports the calculation. If you do know that, if you do have that knowledge explicitly, and we can automate that process of building that knowledge into our systems, I think we have a chance of greatly reducing the cost of mitigation. If we’ve got systems that actually use causal inference to find out what things are possible proxies, and turn them into knowledge, then we have the likelihood of being able to build systems that are far better at this than we humans are. And I think that’s a very, very hopeful point of view. And there is a movement at the moment from being data-centric to being data- and knowledge-centric. People have been saying that data is the new oil, and I’d say that that’s exactly right. It’s something that you use a lot when your technology is primitive, but it’s not really all that good for the world. Knowledge-based systems are the new wind and solar and we should move towards them as soon as possible. And particularly, systems that have explicit knowledge about what the right thing to do is.

27:40 Raluca Crisan:

We’d love to see more industry looking at it in a serious way – I mean, people are looking at it in a very serious way, but I mean looking at it more for these purposes. What else… It’s the people that build the system that is super important, not because they’re the only important ones – it’d be great to get the consumers somehow to contribute to this system, but we’re not there yet. People want to build better systems, especially people who build stuff – they want to build better stuff. So it’s not a problem. It’s just a question of getting them some tools like research and so forth, so they can actually do that easily.

28:40 Susanna Raj:

Thank you, thank you, both of you, and I really enjoyed this discussion, especially on ‘data is the new oil’. I think that actually, the answer is in the sentence itself. Like Michael pointed out we are supposed to use them only for a limited period of time until we find something better and that ‘better’ could be a global depository of knowledge; of understanding of cultures and languages and everything together. That is where we need to be moving towards, even though that’s a harder problem. But I think the industry as a whole needs to invest in it and move towards that. With that, I would like to wrap up and give the stage back to Shilpi and thank you both Michael and Raluca for coming on the show.

In this Panel discussion members of the DataEthics4All Leadership team discuss the topics of Vaccine Mandates and the Great Resignation and discuss whether vaccines should be mandated and the impact COVID-19 has had on the jobs market and the decision more and more of us are making to e.g. make that career change, or go it alone as an entrepreneur or start up.