There is no doubt that machine learning AIs have impressed us with their superhuman performance in the last few years. Computer programs such as Alphago, Alphazero, and Openai Five have shown that machines can dominate us humans in Go, Chess and Dota2. While these advances in technology are making a lot of people excited, they are also making some people very scared. If machines can beat us mano a mano in some of our most complex games, what is stopping them from dominating us in real life? In an interview with BBC, Stephen Hawkins said that "The development of full artificial intelligence could spell the end of the human race," and during a presentation for MIT Elon Musk called AI our "biggest existential threat." Surely, when Elon Musk and Stephen Hawkins are calling out the dangers of AI, the rest of us should be afraid.



Or should we?

If you are lying awake at night, fearing that your computer is plotting to kill you, then I can assure you that we are nowhere near creating The Terminator just jet. Luckily for us, it turns out that neural networks, the main component of most recent AIs, are extremely easy to trick. With a carefully selected alteration of the input you can fool the AIs into believing anything you want.


Take Google’s Googlenet as an example, a very impressive AI that was able to win the ILSVR2014 competition by correctly classifying 93% of the competition’s 55502 objects from 200 different classes. Many of these classes were difficult to distinguish for non-expert humans such as the Siberian husky and the Eskimo dog that you can see in the image underneath.





You would think that since Googlenet performed well on the images from the competition, which it had never seen before, it should perform well on any image containing the same animals?



Well, no. It doesn’t.



With just a small alteration of an image we can fool the AI into believing it is seeing any of the 199 other classes we want. In the image below, there is a panda to the left which Googlenet managed to classify correctly. This paper shows that with just a small change we can get an image that humans are unable to distinguish from the original, but which completely fools the AI.




As you can see the AI thought the image to the right contained a gibbon with 99.3 % confidence although there is only 0.7% noise added to it. The rightmost image is what we call an adversarial example, an input that tricks a neural network although it is almost completely identical to something a neural network interprets correctly.


Every AI that uses neural networks is vulnerable to adversarial examples and not only those built for images. We see the same phenomenon in natural language processing, speech to text, translation and all other fields that use neural networks. The adversarial examples are also very easy to create, and if you want to try it out yourself you should try this python repo. All you need is the neural network model and you can create adversarial examples for any neural network. If you don’t have access to the neural network model, that is not a problem. Adversarial examples are transferable between different neural networks, as shown here. This means if you train a neural network on a data set you can create adversarial examples to fool any other neural network trained on the same data set. If that is not possible you can do a decision based attack if you have access to the input and output of the neural network.


You might think that adversarial examples are a weird curiosity that does not exist in the real world, but researchers have shown that even if you print a physical copy of an adversarial image and feed it to a neural network, it is still misclassified. You could also alter physical objects to create adversarial examples. The image below is taken from this paper which describes how you can create adversarial examples in the real world with graffiti or stickers.





In the eight years since adversarial examples were discovered, almost every defence created against them that have been analysed by other researchers have been disproven or shown to be incomplete. Those who are not disproven are only technically correct and do not provide any real defence against adversarial examples. The slide below is taken from a lecture by Dr. Carlini, and it shows what the best defence methods against adversarial examples can do.






In the slide we see that we can defend against adversarial examples close to the original input, but not against adversarial examples that are just slightly more altered and still indistinguishable to humans. We also see that these defences have a negative impact on the accuracy of the neural network.




To summarize, there is no effective way for a neural network based AI to defend against attacks that are impossible for a human to recognize and fairly easy to create. I think it is safe to say "hasta la vista, baby" to evil Arnold. At least for now.

What do we say to our robotic overlords? Not today.

Hvordan kan du sette retning for din personlige utvikling? Med bakgrunn i mine egne styrker introduserer jeg en pendelmodell. Denne modellen og mine  eksempler viser en fremgangsmåte for å velge hvordan man ønsker å utvikle seg.

Hei der! Mitt navn er Jarle og jeg har vært i Bekk siden januar 2013. De siste årene har jeg hatt et stadig større fokus på folk og coaching. Jeg brenner for å lykkes og hjelpe andre å lykkes. I denne artikkelserien på tre deler, «Selvutvikling på 1-2-3», ønsker jeg å dele erfaringer, modeller og verktøy som jeg har hatt mye nytte av. Både for min egen del, men også for å coache andre. Dette er den andre av tre artikler i serien. Har du ikke lest den første? Da anbefaler jeg deg å starte med denne lenken.

«I 2021 skal jeg høre mer på meg selv og ikke ta så mye hensyn til hva andre sier». Nyttårsforsettet fikk umiddelbart en reaksjon fra min samboer: «Det synes jeg er en utrolig dårlig ide!». Hun og jeg er ganske forskjellige: Hun er svært opptatt av å la andre komme til, og synes ofte at jeg kan ta for mye plass. Med bakgrunn i min målsetning om å jobbe med arroganse, hvordan ble det slik at jeg skulle høre mer på meg selv og mindre på andre? Er ikke det feil retning?

Det er ingen objektiv fasit for hvordan vi skal være som mennesker. Heldigvis! Det fine med dette er at vi selv kan bestemme målene våre. I min erfaring ligger disse målene nær styrkene våre. Vi liker det vi er gode på, naturlig nok. Med bakgrunn i en ferdig utfylt kjernekvadrant kan vi ta ytterpunktene (overslaget til styrken og allergien) og definere utfallsrommet. La oss se tilbake på eksempelet mitt fra første artikkel:

Jeg ønsker å jobbe med arrogansen, men beholde selvtilliten. Selvtillit og ydmykhet er komplementært motsatte styrker. Overslagene er henholdsvis arroganse og usynlighet. Mulighetsrommet kan derfor grovt defineres til å være avgrenset av disse overslagene: Selvtillit kan ikke «gå forbi» arroganse, og du kan ikke bli mer ydmyk enn at du er helt usynlig.

Jeg ønsker å beholde selvtilliten, den identifiserte styrken. Derfor vet jeg at fasit vil ligge nærmere arroganse enn usynlighet. Jeg vet dog også at overslaget mitt gjør at jeg befinner meg for nærme arroganse og må forsøke å gjøre en korrigering i retning usynlighet. Det er lettere sagt enn gjort å bestemme seg for nøyaktig hvor man skal lande. La meg bruke en pendel for å visualisere hvor jeg er og hvor jeg vil:

I min erfaring er en god fremgangsmåte å la pendelen svinge et godt stykke forbi fasit (ønsket utviklingsområde) i retning av det andre ytterpunktet. Hypotesen er at du kan gjøre dette bevisst frem og tilbake og over tid få mindre og mindre utslag slik at din adferd konvergerer mot ønsket fasit:

Det som er fascinerende med dette er at vi med vilje ønsker å gå mot den allergien vi har for å få til dette. Vi må være nysgjerrige på de som har komplementerende styrker og kanskje spesielt overslagene deres. Fordi dette ikke faller oss naturlig må vi mer eller mindre herme etter dem. For min del var det å aktivt ta lite plass noe jeg hadde fint lite erfaring med. For andre er det «second nature». Ved å observere og diskutere med dem og få innsikt i hvordan de fungerer, kan jeg øve på det samme. Vi kan snu rundt på det og se for oss hvordan en svært ydmyk person plutselig skal lære seg å ta mye plass. Det kommer ikke naturlig. Fordi dette er så uvant for oss blir nødvendigvis også de initielle utslagene store. Jeg mener at vi må øve på dette i reelle situasjoner, ikke bare rollespill. «Teste i prod» for de som kjenner IT-verden.

Bakgrunnen for mitt nyttårsforsett som beskrevet innledningsvis var en rekke situasjoner hvor jeg aktivt hadde gått inn for å gi andre plass, hvor jeg i etterpåklokskap angret på at jeg ikke stolte mer på meg selv og tok styring. Pendelen hadde, som forventet, svingt for langt bort fra min opprinnelige styrke. Da følte jeg at en kurskorreksjon var nødvendig. Gjennom 2021 har jeg fått litt feedback på at jeg har tatt mye plass. Neste steg for min del nå er å sette av tid til å vurdere om jeg er enig i at dette er negativt. All den tid du selv definerer fasit, må du selv evaluere hvor du står. Du kan gjerne bruke innspill fra andre, men jeg føler sterkt på at dette er et målbilde du selv må eie. I denne artikkelen har jeg dypdykket i ett spesifikt personlighetstrekk jeg jobber med. Det er en del av et større målbilde og en helhet vi skal se nærmere på i neste (og siste) artikkel i denne serien. 

Selvutvikling på 1-2-3: 2) Pendelmodellen – sett retning for din personlige utvikling

With more than a year since the Schrems-II verdict, I am wondering if it has really left us with any better privacy, or if it has actually been a net loss for European citizens.

Photo by Daniel Radford on Unsplash

To make a very brief (and probably somewhat unfair) summary: The Schrems-II decision by the European Court of Justice decided that Facebook had broken European law by transferring the personal information of Max Schrems to the United States under the Privacy Shield agreement. This invalidated the agreement, and also set the precedence that the US justice system could not provide adequate protection to the fundamental rights of Europeans.

With this backdrop, the European Data Protection Board (EDPB) issued a set of guidelines to how transfers to the US could be done in the face of the new judgement. A crucial issue then arose: Can you transfer data to a European company if that company in turn is owned by an American company? Will this constitute a transfer to the US, and thus be in breach of the Schrems-II decision?

Those who argue yes, say that the American owners can be forced by the US justice system (and in particular the FISA-courts) to make their European subsidiaries break European law and hand over data that the US seeks. The face that this possibility exists, they argue, is in itself a transfer, and thus in violation of the Schrems-II ruling.

This has caused many businesses to question the use of American owned services such as Google Cloud Platform, Amazon Web Services, and Microsoft Azure, the three leading providers of cloud services. Instead, they may have remained with on-prem solutions, or sought out smaller European providers without American owners.

What's the big deal?

This is most likely not a good thing for Europeans! There are great security and privacy benefits when moving to the large cloud providers. These companies spend enormous amounts of money and resources at keeping their platforms secure. Microsoft has said at one point that they employ 3 500 security professionals and spend more than $1 billion on security alone. That is most likely more than the entire staff and operational budget of most medium sized cloud providers.

Security is a key feature of privacy, and those who have chosen what might be less secure providers in order to mitigate what seems to be a largely theoretical threat might have done what is legally correct, but I would argue that it might be the wrong choice for their customers in the long run.

The chances that a company hosting data with Google in Europe will be the target of a FISA case, that culminates in the court forcing Alphabet Inc. to secretly make their European company break the GDPR and hand out customer information, is extremely low for most users of the platform. In addition, most such cases are much more easily solved by the American government asking the local government in the country of the company that owns the data to use local laws to force the release of the information.

In the meantime, real threats to our privacy are all around us. Both individual hackers, criminal gangs and nation state actors attempt to steal information to publish it or sell it. Every day loads of personal information that should have been kept secret is leaked, and this has real consequences for real people. The time spent trying to wrap our heads around how we can comply with a highly unlikely scenario has most likely cost us the opportunity to protect against the actual threats to our users.

11-Dec

Today's posts

What do we say to our robotic overlords? Not today.

Selvutvikling på 1-2-3: 2) Pendelmodellen – sett retning for din personlige utvikling

Schrems-II – At what cost?