Artificial Intelligence
Security threats in AIs such as ChatGPT revealed by researchers
- University of Sheffield scientists have discovered natural language processing tools (NLP), such as ChatGPT, can be tricked into producing malicious code that could lead to cyber attacks
- Study is the first to demonstrate that NLP models can be exploited to attack real-world computer systems used in a wide range of industries
- Results show AI language models are vulnerable to simple backdoor attacks, such as planting a Trojan Horse, that could be triggered at any time to steal information or bring down services
- Findings also highlight the security risks in how people are using AI tools to learn programming languages to interact with databases
Newswise — Artificial intelligence (AI) tools such as ChatGPT can be tricked into producing malicious code, which could be used to launch cyber attacks, according to research from the University of Sheffield.
The study, by academics from the University’s Department of Computer Science, is the first to demonstrate that Text-to-SQL systems – AI that enables people to search databases by asking questions in plain language and are used throughout a wide range of industries – can be exploited to attack computer systems in the real world.
Findings from the research have revealed how the AIs can be manipulated to help steal sensitive personal information, tamper with or destroy databases, or bring down services through Denial-of-Service attacks.
As part of the study, the Sheffield academics found security vulnerabilities in six commercial AI tools and successfully attacked each one.
The AI tools they studied were:
- BAIDU-UNIT – a leading Chinese intelligent dialogue platform adopted by high-profile clients in many industries, including e-commerce, banking, journalism, telecommunications, automobile and civil aviation
- ChatGPT
- AI2SQL
- AIHELPERBOT
- Text2SQL
- ToolSKE
The researchers found that if they asked each of the AIs specific questions, they produced malicious code. Once executed, the code would leak confidential database information, interrupt a database’s normal service, or even destroy it. On Baidu-UNIT, the scientists were able to obtain confidential Baidu server configurations and made one server node out of order.
Xutan Peng, a PhD student at the University of Sheffield, who co-led the research, said: “In reality many companies are simply not aware of these types of threats and due to the complexity of chatbots, even within the community, there are things that are not fully understood.
“At the moment, ChatGPT is receiving a lot of attention. It’s a standalone system, so the risks to the service itself are minimal, but what we found is that it can be tricked into producing malicious code that can do serious harm to other services.”
Findings from the study also highlight the dangers in how people are using AI to learn programming languages, so they can interact with databases.
Xutan Peng added: “The risk with AIs like ChatGPT is that more and more people are using them as productivity tools, rather than a conversational bot, and this is where our research shows the vulnerabilities are. For example, a nurse could ask ChatGPT to write an SQL command so that they can interact with a database, such as one that stores clinical records. As shown in our study, the SQL code produced by ChatGPT in many cases can be harmful to a database, so the nurse in this scenario may cause serious data management faults without even receiving a warning.”
As part of the study, the Sheffield team also discovered it’s possible to launch simple backdoor attacks, such as planting a “Trojan Horse” in Text-to-SQL models by poisoning the training data. Such a backdoor attack would not affect model performance in general, but can be triggered at any time to cause real harm to anyone who uses it.
Dr Mark Stevenson, a Senior Lecturer in the Natural Language Processing research group at the University of Sheffield, said: “Users of Text-to-SQL systems should be aware of the potential risks highlighted in this work. Large language models, like those used in Text-to-SQL systems, are extremely powerful but their behaviour is complex and can be difficult to predict. At the University of Sheffield we are currently working to better understand these models and allow their full potential to be safely realised.”
The Sheffield researchers presented their paper at ISSRE – a major academic and industry conference for software engineering earlier this month (10 October 2023). They are working with stakeholders across the cybersecurity community to address the vulnerabilities, as Text-to-SQL systems continue to be more widely used throughout society.
Their work has already been recognised by Baidu whose Security Response Centre officially rated the vulnerabilities as ‘Highly Dangerous’. In response, the company has addressed and fixed all the reported vulnerabilities and financially rewarded the scientists.
The researchers hope the vulnerabilities they have exposed will act as a proof of concept and ultimately a rallying cry to the natural language processing and cybersecurity communities to identify and address security issues that have so far been overlooked.
Xutan Peng added: “Our efforts are being recognised by industry and they are following our advice to fix these security flaws. However, we are opening a door on an endless road – what we now need to see are large groups of researchers creating and testing patches to minimise security risks through open source communities.
“There will always be more advanced strategies being developed by attackers, which means security strategies must keep pace. To do so we need a new community to fight these next generation attacks.”
Journal Link: The 34th IEEE International Symposium on Software Reliability Engineering
Source: University of Sheffield
Author
Discover more from Daily News
Subscribe to get the latest posts sent to your email.
Tech
How close are quantum computers to being really useful? Podcast
Quantum computers could revolutionize science by solving complex problems. However, scaling and error correction remain significant challenges before achieving practical applications.
Quantum computers have the potential to solve big scientific problems that are beyond the reach of today’s most powerful supercomputers, such as discovering new antibiotics or developing new materials.
But to achieve these breakthroughs, quantum computers will need to perform better than today’s best classical computers at solving real-world problems. And they’re not quite there yet. So what is still holding quantum computing back from becoming useful?
In this episode of The Conversation Weekly podcast, we speak to quantum computing expert Daniel Lidar at the University of Southern California in the US about what problems scientists are still wrestling with when it comes to scaling up quantum computing, and how close they are to overcoming them.
Quantum computers harness the power of quantum mechanics, the laws that govern subatomic particles. Instead of the classical bits of information used by microchips inside traditional computers, which are either a 0 or a 1, the chips in quantum computers use qubits, which can be both 0 and 1 at the same time or anywhere in between. Daniel Lidar explains:
“Put a lot of these qubits together and all of a sudden you have a computer that can simultaneously represent many, many different possibilities … and that is the starting point for the speed up that we can get from quantum computing.”
Faulty qubits
One of the biggest problems scientist face is how to scale up quantum computing power. Qubits are notoriously prone to errors – which means that they can quickly revert to being either a 0 or a 1, and so lose their advantage over classical computers.
Scientists have focused on trying to solve these errors through the concept of redundancy – linking strings of physical qubits together into what’s called a “logical qubit” to try and maximise the number of steps in a computation. And, little by little, they’re getting there.
In December 2024, Google announced that its new quantum chip, Willow, had demonstrated what’s called “beyond breakeven”, when its logical qubits worked better than the constituent parts and even kept on improving as it scaled up.
Lidar says right now the development of this technology is happening very fast:
“For quantum computing to scale and to take off is going to still take some real science breakthroughs, some real engineering breakthroughs, and probably overcoming some yet unforeseen surprises before we get to the point of true quantum utility. With that caution in mind, I think it’s still very fair to say that we are going to see truly functional, practical quantum computers kicking into gear, helping us solve real-life problems, within the next decade or so.”
Listen to Lidar explain more about how quantum computers and quantum error correction works on The Conversation Weekly podcast.
This episode of The Conversation Weekly was written and produced by Gemma Ware with assistance from Katie Flood and Mend Mariwany. Sound design was by Michelle Macklem, and theme music by Neeta Sarl.
Clips in this episode from Google Quantum AI and 10 Hours Channel.
You can find us on Instagram at theconversationdotcom or via e-mail. You can also subscribe to The Conversation’s free daily e-mail here.
Listen to The Conversation Weekly via any of the apps listed above, download it directly via our RSS feed or find out how else to listen here.
Gemma Ware, Host, The Conversation Weekly Podcast, The Conversation
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Author
Discover more from Daily News
Subscribe to get the latest posts sent to your email.
Leo Porter, University of California, San Diego and Daniel Zingaro, University of Toronto
What do you think there are more of: professional computer programmers or computer users who do a little programming?
It’s the second group. There are millions of so-called end-user programmers. They’re not going into a career as a professional programmer or computer scientist. They’re going into business, teaching, law, or any number of professions – and they just need a little programming to be more efficient. The days of programmers being confined to software development companies are long gone.
If you’ve written formulas in Excel, filtered your email based on rules, modded a game, written a script in Photoshop, used R to analyze some data, or automated a repetitive work process, you’re an end-user programmer.
As educators who teach programming, we want to help students in fields other than computer science achieve their goals. But learning how to program well enough to write finished programs can be hard to accomplish in a single course because there is so much to learn about the programming language itself. Artificial intelligence can help.
Lost in the weeds
Learning the syntax of a programming language – for example, where to place colons and where indentation is required – takes a lot of time for many students. Spending time at the level of syntax is a waste for students who simply want to use coding to help solve problems rather than learn the skill of programming.
As a result, we feel our existing classes haven’t served these students well. Indeed, many students end up barely able to write small functions – short, discrete pieces of code – let alone write a full program that can help make their lives better.
Tools built on large language models such as GitHub Copilot may allow us to change these outcomes. These tools have already changed how professionals program, and we believe we can use them to help future end-user programmers write software that is meaningful to them.
These AIs almost always write syntactically correct code and can often write small functions based on prompts in plain English. Because students can use these tools to handle some of the lower-level details of programming, it frees them to focus on bigger-picture questions that are at the heart of writing software programs. Numerous universities now offer programming courses that use Copilot.
At the University of California, San Diego, we’ve created an introductory programming course primarily for those who are not computer science students that incorporates Copilot. In this course, students learn how to program with Copilot as their AI assistant, following the curriculum from our book. In our course, students learn high-level skills such as decomposing large tasks into smaller tasks, testing code to ensure its correctness, and reading and fixing buggy code.
Freed to solve problems
In this course, we’ve been giving students large, open-ended projects and couldn’t be happier with what they have created.
For example, in a project where students had to find and analyze online datasets, we had a neuroscience major create a data visualization tool that illustrated how age and other factors affected stroke risk. Or, for example, in another project, students were able to integrate their personal art into a collage, after applying filters that they had created using the programming language Python. These projects were well beyond the scope of what we could ask students to do before the advent of large language model AIs.
Given the rhetoric about how AI is ruining education by writing papers for students and doing their homework, you might be surprised to hear educators like us talking about its benefits. AI, like any other tool people have created, can be helpful in some circumstances and unhelpful in others.
In our introductory programming course with a majority of students who are not computer science majors, we see firsthand how AI can empower students in specific ways – and promises to expand the ranks of end-user programmers.
Leo Porter, Teaching Professor of Computer Science and Engineering, University of California, San Diego and Daniel Zingaro, Associate Professor of Mathematical and Computational Sciences, University of Toronto
This article is republished from The Conversation under a Creative Commons license. Read the original article.
Discover more from Daily News
Subscribe to get the latest posts sent to your email.
Tech
From shrimp Jesus to fake self-portraits, AI-generated images have become the latest form of social media spam
Renee DiResta, Stanford University; Abhiram Reddy, Georgetown University, and Josh A. Goldstein, Georgetown University
Suppose you’ve spent time on Facebook over the past six months. In that case, you may have noticed photorealistic images that are too good to be true: children holding paintings that look like the work of professional artists, or majestic log cabin interiors that are the stuff of Airbnb dreams.
Others, such as renderings of Jesus made out of crustaceans, are just bizarre.
Like the AI image of the pope in a puffer jacket that went viral in May 2023, these AI-generated images are increasingly prevalent – and popular – on social media platforms. Even as many of them border on the surreal, they’re often used to bait engagement from ordinary users.
Our team of researchers from the Stanford Internet Observatory and Georgetown University’s Center for Security and Emerging Technology investigated over 100 Facebook pages that posted high volumes of AI-generated content. We published the results in March 2024 as a preprint paper, meaning the findings have not yet gone through peer review.
We explored patterns of images, unearthed evidence of coordination between some of the pages, and tried to discern the likely goals of the posters.
Page operators seemed to be posting pictures of AI-generated babies, kitchens or birthday cakes for a range of reasons.
There were content creators innocuously looking to grow their followings with synthetic content; scammers using pages stolen from small businesses to advertise products that don’t seem to exist; and spammers sharing AI-generated images of animals while referring users to websites filled with advertisements, which allow the owners to collect ad revenue without creating high-quality content.
Our findings suggest that these AI-generated images draw in users – and Facebook’s recommendation algorithm may be organically promoting these posts.
Generative AI meets scams and spam
Internet spammers and scammers are nothing new.
For more than two decades, they’ve used unsolicited bulk email to promote pyramid schemes. They’ve targeted senior citizens while posing as Medicare representatives or computer technicians.
On social media, profiteers have used clickbait articles to drive users to ad-laden websites. Recall the 2016 U.S. presidential election, when Macedonian teenagers shared sensational political memes on Facebook and collected advertising revenue after users visited the URLs they posted. The teens didn’t care who won the election. They just wanted to make a buck.
In the early 2010s, spammers captured people’s attention with ads promising that anyone could lose belly fat or learn a new language with “one weird trick.”
AI-generated content has become another “weird trick.”
It’s visually appealing and cheap to produce, allowing scammers and spammers to generate high volumes of engaging posts. Some of the pages we observed uploaded dozens of unique images per day. In doing so, they followed Meta’s own advice for page creators. Frequent posting, the company suggests, helps creators get the kind of algorithmic pickup that leads their content to appear in the “Feed,” formerly known as the “News Feed.”
Much of the content is still, in a sense, clickbait: Shrimp Jesus makes people pause to gawk and inspires shares purely because it is so bizarre.
Many users react by liking the post or leaving a comment. This signals to the algorithmic curators that perhaps the content should be pushed into the feeds of even more people.
Some of the more established spammers we observed, likely recognizing this, improved their engagement by pivoting from posting URLs to posting AI-generated images. They would then comment on the post of the AI-generated images with the URLs of the ad-laden content farms they wanted users to click.
But more ordinary creators capitalized on the engagement of AI-generated images, too, without obviously violating platform policies.
Rate ‘my’ work!
When we looked up the posts’ captions on CrowdTangle – a social media monitoring platform owned by Meta and set to sunset in August – we found that they were “copypasta” captions, which means that they were repeated across posts.
Some of the copypasta captions baited interaction by directly asking users to, for instance, rate a “painting” by a first-time artist – even when the image was generated by AI – or to wish an elderly person a happy birthday. Facebook users often replied to AI-generated images with comments of encouragement and congratulations
Algorithms push AI-generated content
Our investigation noticeably altered our own Facebook feeds: Within days of visiting the pages – and without commenting on, liking or following any of the material – Facebook’s algorithm recommended reams of other AI-generated content.
Interestingly, the fact that we had viewed clusters of, for example, AI-generated miniature cow pages didn’t lead to a short-term increase in recommendations for pages focused on actual miniature cows, normal-sized cows or other farm animals. Rather, the algorithm recommended pages on a range of topics and themes, but with one thing in common: They contained AI-generated images.
In 2022, the technology website Verge detailed an internal Facebook memo about proposed changes to the company’s algorithm.
The algorithm, according to the memo, would become a “discovery-engine,” allowing users to come into contact with posts from individuals and pages they didn’t explicitly seek out, akin to TikTok’s “For You” page.
We analyzed Facebook’s own “Widely Viewed Content Reports,” which lists the most popular content, domains, links, pages and posts on the platform per quarter.
It showed that the proportion of content that users saw from pages and people they don’t follow steadily increased between 2021 and 2023. Changes to the algorithm have allowed more room for AI-generated content to be organically recommended without prior engagement – perhaps explaining our experiences and those of other users.
‘This post was brought to you by AI’
Since Meta currently does not flag AI-generated content by default, we sometimes observed users warning others about scams or spam AI content with infographics.
Meta, however, seems to be aware of potential issues if AI-generated content blends into the information environment without notice. The company has released several announcements about how it plans to deal with AI-generated content.
In May 2024, Facebook will begin applying a “Made with AI” label to content it can reliably detect as synthetic.
But the devil is in the details. How accurate will the detection models be? What AI-generated content will slip through? What content will be inappropriately flagged? And what will the public make of such labels?
While our work focused on Facebook spam and scams, there are broader implications.
Reporters have written about AI-generated videos targeting kids on YouTube and influencers on TikTok who use generative AI to turn a profit.
Social media platforms will have to reckon with how to treat AI-generated content; it’s certainly possible that user engagement will wane if online worlds become filled with artificially generated posts, images and videos.
Shrimp Jesus may be an obvious fake. But the challenge of assessing what’s real is only heating up.
Renee DiResta, Research Manager of the Stanford Internet Observatory, Stanford University; Abhiram Reddy, Research Assistant at the Center for Security and Emerging Technology, Georgetown University, and Josh A. Goldstein, Research Fellow at the Center for Security and Emerging Technology, Georgetown University
This article is republished from The Conversation under a Creative Commons license. Read the original article.
The science section of our news blog STM Daily News provides readers with captivating and up-to-date information on the latest scientific discoveries, breakthroughs, and innovations across various fields. We offer engaging and accessible content, ensuring that readers with different levels of scientific knowledge can stay informed. Whether it’s exploring advancements in medicine, astronomy, technology, or environmental sciences, our science section strives to shed light on the intriguing world of scientific exploration and its profound impact on our daily lives. From thought-provoking articles to informative interviews with experts in the field, STM Daily News Science offers a harmonious blend of factual reporting, analysis, and exploration, making it a go-to source for science enthusiasts and curious minds alike. https://stmdailynews.com/category/science/
Author
Discover more from Daily News
Subscribe to get the latest posts sent to your email.
-
Urbanism1 year agoSignal Hill, California: A Historic Enclave Surrounded by Long Beach
-
News2 years agoDiana Gregory Talks to us about Diana Gregory’s Outreach Services
-
Senior Pickleball Report2 years agoThe Absolute Most Comfortable Pickleball Shoe I’ve Ever Worn!
-
STM Blog2 years agoWorld Naked Gardening Day: Celebrating Body Acceptance and Nature
-
Senior Pickleball Report2 years agoACE PICKLEBALL CLUB TO DEBUT THEIR HIGHLY ANTICIPATED INDOOR PICKLEBALL FRANCHISES IN THE US, IN EARLY 2023
-
Travel2 years agoUnique Experiences at the CitizenM
-
Automotive2 years ago2023 Nissan Sentra pricing starts at $19,950
-
Senior Pickleball Report2 years ago“THE PEOPLE’S CHOICE AWARDS OF PICKLEBALL” – VOTING OPEN