What is natural language processing? | Definition from TechTarget
"Ethics At Arm's Length": Kate Crawford Book Excerpt
Whether it's online translation tools arbitrarily assigning genders to certain professions or facial recognition software misidentifying black faces, there are numerous examples these days of bias generated by artificial intelligence (AI). Most AI systems reflect characteristics of the dominant voice in their code and in the data they use to learn, and that is male and white. But this process of bias didn't just show up overnight, it's the result of decades of developments in the field, as researchers slowly disconnected with the subjects they were investigating according to Australian AI expert Kate Crawford. It's a process that creatives have recently begun using to inspire their art, as people such as Caroline Sinders and Joy Buolamwini are showing. In her new book Atlas of AI, Crawford examines the burgeoning growth of AI in an environment where "ethical questions (are separated) away from the technical." The following excerpt from her book is entitled "Ethics at arm's length."-
© Yale University Press The great majority of university-based AI research is done without any ethical review process. But if machine learning techniques are being used to inform decisions in sensitive domains like education and health care, then why are they not subject to greater review? To understand that, we need to look at the precursor disciplines of artificial intelligence. Before the emergence of machine learning and data science, the fields of applied mathematics, statistics, and computer science had not historically been considered forms of research on human subjects. In the early decades of AI, research using human data was usually seen to be a minimal risk. Even though datasets in machine learning often come from and represent people and their lives, the research that used those datasets was seen more as a form of applied math with few consequences for human subjects. The infrastructures of ethics protections, like university-based institutional review boards (IRBs), had accepted this position for years. This initially made sense; IRBs had been overwhelmingly focused on the methods common to biomedical and psychological experimentation in which interventions carry clear risks to individual subjects. Computer science was seen as far more abstract. Once AI moved out of the laboratory contexts of the 1980s and 1990s and into real-world situations—such as attempting to predict which criminals will reoffend or who should receive welfare benefits—the potential harms expanded. Further, those harms affect entire communities as well as individuals. But there is still a strong presumption that publicly available data sets pose minimal risks and therefore should be exempt from ethics review. This idea is the product of an earlier era, when it was harder to move data between locations and very expensive to store it for long periods. Those earlier assumptions are out of step with what is currently going on in machine learning. Now datasets are more easily connectable, indefinitely repurposable, continuously updatable, and frequently removed from the context of collection. 
Once AI moved out of the laboratory and into real-world situations the potential harms expandedPhoto: Jürgen Jester / Unsplash The risk profile of AI is rapidly changing as its tools become more invasive and as researchers are increasingly able to access data without interacting with their subjects. For example, a group of machine learning researchers published a paper in which they claimed to have developed an "automatic system for classifying crimes." In particular, their focus was on whether a violent crime was gang-related, which they claimed their neural network could predict with only four pieces of information: the weapon, the number of suspects, the neighbourhood, and the location. They did this using a crime dataset from the Los Angeles Police Department, which included thousands of crimes that had been labelled by police as gang-related. Gang data is notoriously skewed and riddled with errors, yet researchers use this database and others like it as a definitive source for training predictive AI systems. The CalGang database, for example, which is widely used by police in California, has been shown to have major inaccuracies. The state auditor discovered that 23 percent of the hundreds of records it reviewed lacked adequate support for inclusion. The database also contained forty-two infants, twenty-eight of whom were listed for having "admitting to being gang members." Most of the adults on the list had never been charged, but once they were included in the database, there was no way to have their name removed. Reasons for being included might be as simple as chatting with a neighbour while wearing a red shirt; using these trifling justifications, Black and Latinx people have been disproportionately added to the list.
The CalGang database, which is widely used by police in California, has been shown to have major inaccuraciesCredit: Sean Lee / Unsplash When the researchers presented their gang-crime prediction project at a conference, some attendees were troubled. As reported by Science, questions from the audience included, "How could the team be sure the training data were not biased to begin with?" and "What happens when someone is mislabelled as a gang member?" Hau Chan, a computer scientist now at Harvard University who presented the work, responded that he couldn't know how the new tool would be used. "[These are the] sort of ethical questions that I don't know how to answer appropriately," he said, being just "a researcher." An audience member replied by quoting a lyric from Tom Lehrer's satiric song about the wartime rocket scientist Wernher von Braun: "Once the rockets are up, who cares where they come down?" This separation of ethical questions away from the technical reflects a wider problem in the field, where the responsibility for harm is either not recognized or seen as beyond the scope of the research. As Anna Lauren Hoffman writes: "The problem here isn't only one of biased datasets or unfair algorithms and of unintended consequences. It's also indicative of a more persistent problem of researchers actively reproducing ideas that damage vulnerable communities and reinforce current injustices. Even if the Harvard team's proposed system for identifying gang violence is never implemented, hasn't a kind of damage already been done? Wasn't their project an act of cultural violence in itself?" Sidelining issues of ethics is harmful in itself, and it perpetuates the false idea that scientific research happens in a vacuum, with no responsibility for the ideas it propagates. 
AI scientist Joseph Weizenbaum wrote in 1976 that computer science was already seeking to circumvent all human contextsCredit: Chris Barbalis / Unsplash The reproduction of harmful ideas is particularly dangerous now that AI has moved from being an experimental discipline used only in laboratories to being tested at scale on millions of people. Technical approaches can move rapidly from conference papers to being deployed in production systems, where harmful assumptions can become ingrained and hard to reverse.
Machine learning and data-science methods can create an abstract relationship between researchers and subjects, where work is being done at a distance, removed from the communities and individuals at risk of harm. This arm's-length relationship of AI researchers to the people whose lives are reflected in datasets is a long-established practice. Back in 1976, when AI scientist Joseph Weizenbaum wrote his scathing critique of the field, he observed that computer science was already seeking to circumvent all human contexts. He argued that data systems allowed scientists during wartime to operate at a psychological distance from the people "who would be maimed and killed by the weapons systems that would result from the ideas they communicated." The answer, in Weizenbaum's view, was to directly contend with what data actually represents: "The lesson, therefore, is that the scientist and technologist must, by acts of will and of the imagination, actively strive to reduce such psychological distances, to counter the forces that tend to remove him from the consequences of his actions. He must — it is as simple as this — think of what he is actually doing." 
Kate CrawfordPhoto: Cath Muscat Weizenbaum hoped that scientists and technologists would think more deeply about the consequences of their work — and of who might be at risk. But this would not become the standard of the AI field. Instead, data is more commonly seen as something to be taken at will, used without restriction, and interpreted without context. There is a rapacious international culture of data harvesting that can be exploitative and invasive and can produce lasting forms of harm. And there are many industries, institutions, and individuals who are strongly incentivized to maintain this colonizing attitude — where data is there for the taking—and they do not want it questioned or regulated. This excerpt from Atlas of AI is re-printed here with the kind permission of Kate Crawford and Yale University Press. The book can be purchased here.
May 2021
Do you have questions about this article? Write to us.
27 Rights Groups Demand Zoom Abandon 'Invasive,' And 'Inherently Biased' Emotion Recognition Software
More than two dozen rights groups are calling on Zoom to scrap its efforts to explore controversial emotion recognition technology. The pushback from 27 separate groups represents some of the most forceful resistance to the emerging tech yet, which critics fear remains inaccurate and under tested.
In an open letter addressed to Zoom CEO Co-Founder, Eric S. Yuan, the groups led by Fight for the Future criticized the company's alleged emotional data mining efforts as, "a violation of privacy and human rights." The letter takes aim at the tech which it described as "inherently biased," against non-white individuals.
The groups challenged Zoom to lean into its role as the industry leader in video conferencing to set standards which other smaller companies might follow. "You can make it clear that this technology has no place in video communication," the letter reads.
"If Zoom advances with these plans, this feature will discriminate against people of certain ethnicities and people with disabilities, hardcoding stereotypes into millions of devices," Fight for the Future Director of Campaign and Operation Caitlin Seeley George said. "Beyond mining users for profit and allowing businesses to capitalize on them, this technology could take on far more sinister and punitive uses."
Though emotion recognition technology has simmered in tech incubators for years, it's more recently gained renewed interest among major consumer facing tech companies like Zoom. Earlier this year, Zoom revealed its interest in the tech to Protocol, claiming it has active research on how to incorporate emotion AI. In the near term, the company reportedly plans to roll out a feature called Zoom IQ for Sales which will provide meeting hosts with a post meeting sentiment analysis which would try to determine the level of engagement from particular members.
Zoom did not immediately respond to Gizmodo's request for comment.
In her recent book Atlas of AI, USC Annenberg Research Professor Kate Crawford described emotion recognition, also referred to as, "affect recognition" as a type of offshoot of facial recognition. While the latter, more well known system attempts to identify a particular person, affect or emotion recognition aims to, "detect and classify emotions by analyzing any face." Crawford argues there's little evidence current systems can meaningfully make that premise a reality.
"The difficulty in automating the connection between facial movements and basic emotional categories leads to the larger question of whether emotions can be adequately grouped into a small number of discrete categories at all," Crawford writes. "There is the stubborn issue that facial expressions may indicate little about our honest interior states, as anyone who has smiled without feeling truly happy can confirm."
Those concerns have not been enough to stop tech giants from experimenting with the tech, with Intel even reportedly trying to use the tools in virtual classroom settings. There's plenty of potential money to be made in this space as well. Recent global forecasts on emotion detection and recognition software predict the industry could be worth $56 billion by 2024.
"Our emotional states and our innermost thoughts should be free from surveillance," Access Now Senior Policy Analyst Daniel Leufer, said in a statement. "Emotion recognition software has been shown again and again to be unscientific, simplistic rubbish that discriminates against marginalized groups, but even if it did work, and could accurately identify our emotions, it's not something that has any place in our society, and certainly not in our work meetings, our online lessons, and other human interactions that companies like Zoom provide a platform for."
In their letter the rights groups echoed concerns voiced by academics and argued emotion recognition tech in its current state is "discriminatory," and "based off of pseudoscience." They also warned of potentially dangerous unforeseen consequences linked to the tech's rushed rollout.
"The use of this bad technology could be dangerous for students, workers, and other users if their employers, academic or other institutions decide to discipline them for 'expressing the wrong emotions,' based on the determinations of this AI technology," the letter reads.
Still, the rights groups attempted to extend an olive branch and praised Zoom for its past efforts on integrating end-to-end encryption to video call and its decision to remove attendee attention tracking.
"This is another opportunity to show you care about your users and your reputation," the groups wrote. "Zoom is an industry leader, and millions of people are counting on you to steward our virtual future. As a leader, you also have the responsibility of setting the course for other companies in the space."
open article in new tabTime's 100 Most Important People In AI Focuses On The Big CEOs... Plus Grimes
A good way to get the full scope of the current debate surrounding AI is to know the people involved. Time magazine, in all its wisdom, now has its own list of the top 100 figures most important to the AI debate to help us all get to know the players. There are a hell of a lot of AI founders on there, but despite that, there's a good representation of folks on the fringes who could become main characters in the era of AI. Oh, and there's also Grimes.
The list is rather extensive and holds up people who are actively involved in AI and the discussions around it, as well as those on the periphery who may be the catalysts for catapulting AI into even greater popularity—or dragging it back down to Earth. Like all Time's lists, it does have a heavy corporate bent. The "Leaders" category is festooned with major tech companies, with Anthropic heads Dario and Daniela Amodei leading the section by virtue of having a last name that begins with the letter "A." The list tries to tout Anthropic's "Constitution" for AI as leading the discussion for ethical development, even though its main thrust is simply placing 'guardrails' on AI models, something that doesn't really work all the time for even the most sophisticated models.
Most of the who's who list is made up of CEOs—43 in all. The usual suspects are all here. You have OpenAI founder and CEO Sam Altman plus the company's president Greg Brockman. There's all the ultra-rich investors such as Reid Hoffman, Marc Andreessen, plus the big tech heads like Microsoft's chief scientist Jaime Teevan and Google DeepMind CEO Demis Hassabis. Oh, and of course, the early OpenAI investor and now AI coat-tail-riding Elon Musk is there playing catch-up with his nascent startup xAI.
But there's also room for pop culture figures in Time's big list. Music artist Grimes takes a leading role because she's become the figurehead of AI song-making platforms and opened an AI model of her voice for all to use. She has even claimed she would create an album where she faces off against an "AI-hive-mind-collective Grimes." Then there are other artists like Holly Herndon, a singer-songwriter, who has worked longer than Grimes creating music based on the hottest tech of the moment. She's also established a template that allowed artists to opt out of using their work to train AI datasets, which some companies like Stability AI and Hugging Face have complied with.
Time also held up Rootport, the pseudonymous author of the manga titled Cyberpunk: Peach John, as an "Innovator" in AI. The figure has become controversial for his open use of the AI art generator Midjourney to generate the art for the manga. Indeed, the creator has openly said he doesn't know how to draw and has bragged that the work took him just six weeks to finalize. While the work was published by professional publishing house Shinchosha, it's really hard to call anything Rootport's done true "innovation." Other artists have tried creating graphic novels with AI, but they've had little luck getting wider recognition for the art itself, especially from U.S. Copyright laws. On the flip side, there are people like Kelly McKernan in the "Shapers" category who made headlines earlier this year for being one of the few artists to sue Midjourney, along with Stability AI and DeviantArt, for using her art in the AI's training data
The list also finds some rather interesting folks in the entertainment industry. There's Hugo and Nebula award-winning author Ted Chiang, but also Black Mirror creator Charlie Brooker. Lilly Wachowski, the famed co-director of The Matrix, is here. But the list didn't just reference her and her sister Lana's work popularizing a dystopian future war with intelligent machines, she's also used as a figurehead for the ongoing SAG-AFTRA and WGA Hollywood strikes, especially with her early critique of studios using AI "as a tool to generate wealth" at the expense of artists and the culture at large.
And then there's the multitude of common AI critics making an appearance. There's Margaret Mitchell, the chief AI ethics scientist at Hugging Face, and Distributed AI Research Institute founder Timnit Gebru. Both made their names as AI ethicists working for Google before being fired during a larger conversation about AI generating harmful content. There's also University of Washington computational linguistics professor Emily Bender, who is a regular online commentator debunking the hype surrounding AI through her "Stochastic Parrot" thesis.
There was some room left for lesser-known figures. Eighteen-year-old Sneha Revanur, the founder of Encode Justice, got a spot for helping plead with the Biden administration to work quickly on AI regulations. Kate Crawford, an author and professor at USC Annenberg, who wrote in her book Atlas of AI about the labor, environmental, and human costs of the race for AI also nabbed a spot. If there were a big Atlas for all the characters sucked into the whirlpool of the current AI debate, Time might be a good place to start, but it might be better to analyze what those without the billions of dollars in their back pocket have to say about the transformative tech.
open article in new tab
Comments
Post a Comment