Passphrases for Humans: A Cultural Approach to Passphrase Wordlist Generation

okay then we are ready to start again um and i've told flo and scala here that you know there's absolutely no pressure at all uh in this and you know i've told them i'm really looking forward to this talk and they are going to talk about uh passphrase wordless generation and i'm very happy to tell them once again that the author of diceware is in the audience so you know please welcome flo and skylar from perio

sure so um welcome to pass phrases for humans uh so um hi my name is flo florencia haravega i'm the cto at pirio and we are a tiny startup that you may or may not have heard of so my own background is i'm a developer i'm not a security researcher so i'm i'm tackling this problem of passphrases from a sort of developer perspective and a developer who's very interested in user advocacy and user education my name is skyler nagao i am product manager at erio but um mostly kind of a humanities geek and that'll hopefully shine through a little bit and some of the work we're doing here it's just a switch where yeah oh there we go

does that work now oh yeah it does well um so to give you a little brief overview we're gonna the beginning will be some basics we're gonna introduce perio pass phrases and some of the standards that we arrived at uh move towards the problem with some maybe the existing dice word dictionaries discuss then what we consider a more cultural approach to how we can develop these word lists and then uh move to the future and some ideas that we're already working on some ideas that are out there review some current research and then we'd also love to hear from the audience if there's time about their experiences or ideas here for anyone who's not familiar we just

wanted to have a couple of definitions just so we're all on the same page we're going to use some pretty broad ones here passphrase is really just like a password but it's just usually longer and composed of words there are different ways you can do this but we gave two examples on the on the board and some basic ways and entropy is just a lack of predictability to have more entropy is to have less predictability so if you were to guess one number on these dice the 20-sided dice would be harder to guess correctly so for some background what of what the app is for context so we do uh easy to use uh message and

file file sharing with an emphasis on teams um so it's it's cloud-based um and everything is end-to-end encrypted and users need to be able to log in from any device on any platform uh with a passphrase so we derive the user's private key from the passphrase using blade2s and script and um basically so what that means is we don't ever get the private keys so if the user loses the passphrase they're screwed so the phrases are a very important component of actually being able to use the software at all um so when we say that we want this to be easy to use uh we want users to have to to be allowed to not think very much um so

that should also mean that the software should be very hard to screw up so for example that's one of the reasons we're doing end-to-end encryption by default so that you can't accidentally reply to something and send it in plain text um there's actually no way to send stuff unencrypted so there's some stuff though where there is user input required such as choosing a passphrase and so that's where things get really dicey in terms of enforcing some sort of sane default um so minimum password strength is a big question mark for us um so so for the sake of of our research and our our development style there's a couple of assumptions that we're making and we get that these are controversial

in some ways but but this is just kind of what we're rolling with so the first thing is that people cannot and will not remember more than a few passwords so for example i know that i have several hundred passwords there's no way i can memorize several hundred passwords um we also think that password managers are great for most people in most use cases precisely because you you have a million services and you have to be able to access them and have good passwords for them and we also think that critical services should have strong passwords so that includes so that say let's say that maybe you have a password manager like one password and

you have a great passphrase for that and maybe you also have a great passphrase that you memorized for your like gmail and your facebook which you might need to access from someone else's computer from time to time if your phone is dead or whatever so um so based on these assumptions uh when we're deciding on defaults um we have a bit of a trade-off to make so there's there's things that are there's there's some stuff that's more secure but maybe it's less convenient and we need to constantly straddle the line and so that's basically what we do every single day is say okay well this thing is the more secure thing but it's also less convenient and then we we have

to kind of extrapolate from there and sort of imagine what kind of scenarios uh most users will be encountering and pick some sort of same default from there and then also figure out what kind of configurations we want to allow users to define on their own so in terms of of modeling threats um it's it's a bit of a you know it's a bit of an exercise in fantasy but it's also fun so um at what point does trying to crack a password become unreasonable for an attacker so that that depends on who the attacker is and what how much time and money they have so does your does your attacker have five dollars a thousand

dollars maybe one million dollars um it turns out that in certain situations you do get someone who has a million dollars to crack something so like the fbi for example um and if you look at you know like the biggest budgets that you can possibly find um that's kind of what you're looking at um and then there's this wonderful paper uh from joseph bono and stuart schechter that kind of takes a look at gigantic hashing operations um and in how much they cost and sort of extrapolates from there that something like performing a 2 to the power of 80 hashes in a year would cost around 1 billion for a centralized actor sort of based on what you see in bitcoin which

isn't a centralized actor but is in fact distributed and you know and not optimized in various ways so that's that's like really really high end um and so we decided we will take a high-end standard and so we call it the billion dollar standard um so a billion and a full year is probably more money in time than like anyone is willing to spend on cracking a password so if we go up beyond that standard we start getting we start having really really high entropy requirements which is going to mean like really crazy crazy passwords or passphrases um which really affects usability and also like realistically you know we all know that um you know if someone's out to get you

that the cracking a password is not where they're going to spend all their time and money because there's going to be way easier things that they can do um even if you do have billions of dollars at your disposal so um so therefore we arrive at this at this entropy requirement which is um based on on the way that we generate our keys we're using public key stretching a private key stretching sorry with um with script and so we actually end up uh needing in order to get 81-bit passwords which are our one billion dollar passwords we actually need a passphrase with a 67.5 bits of entropy and then we stretch that uh with script um with the

the basically the most amount of script that we can manage that on a slow phones processor it doesn't take like ages to actually sign into your account um so yeah um so from there um i'll let skyler take over so uh it may not mean much to uh everyone i guess but uh to start so our goal is to hit 67.5 bits from our users and the problem right is that most people are really terrible at making passwords like i don't need to iterate too much on this two little stats uh one study found 30 percent of users were using one of the most 10 000 common passwords ashley madison um you know you would think

would be a very maybe more privacy minded service yeah it turned out hundreds of thousands of people were using passwords like those that you see on the board and like one two three four five six um a data dump from gmail in 2014 revealed that the average password strength was roughly like 21.6 bits you know so this is quite far off um from our requirements and unfortunately it's not just like some you know joe schmo whatever who's writing the uh creating these passwords like in that same dump were passwords from you know the companies you see on the board we have journalists from major news networks a division chief at the department of state a paypal senior engineer

and you know it's like oh no surprise that like the github developer has a 96 bit password but all these other people who you know uh all of us are probably trusting with our data you know may not be doing as well here so you know we have this question like well how do we ensure our users are meeting the standard and i mean one thing would be to use things like password meters this is one of my favorites the passive aggressive password meter where as you're entering characters it'll tell you just how bad your password actually is um before before we continue so we were given some little treats slash booze before starting and told that we can

give them out for however we please really so since we're going to be discussing this does anyone know how to pronounce this and i refuse to accept anyone who just tells me each individual letter so whoever pronounces this for us gets this pronounced

we can run with that sure yeah sure i that's i've been calling it zik's been this whole time um we can call it the dropbox method though so you know one problem with these like entropy estimates is like the well the traditional method is like well okay how many different characters are there and then multiply that by how many characters are in the password and then you might see numbers like this it's like oh cool like i can have like leap password and i'm like almost at this requirement right well you know this dropbox method is like actually nah that doesn't really fly because you know um it accounts for some human elements like people's tendency to use these common

keyboard patterns or people's tendency to try to work around password rules in the laziest ways possible like apparently one out of four people will basically just add one to the end of their password yes some people are either really creative really lazy or really egoistic and just use their name for both username and password i don't know if you all can see this because it's kind of small but here's like a breakdown of what sixpen dropbox method looks like where you know we for fun we entered a password that would pass most standards it is baseball one two three written in lead so with mixed case numbers and such and you can see that it actually detects like oh you

know what use the word baseball which happens to for some reason in america be the ninth most common word appearing in passwords it'd recognize the the letter substitutions and these common sequences so although this would pass many standards it turns out to be something that would be cracked uh instantly so our first solution was kind of like a classic lazy developer solution which was like oh okay so there's this tool that we can just import and we'll just we we realized that maybe this is not quite good enough like the the estimates that it's giving or maybe a little off um so okay let's just require 100 bit passphrases and then we'll like write all this intro text about how to choose

a good passphrase and we'll give a strength meter and that's totally a great solution and users will totally do the right thing and it'll be great and then of course it wasn't um so basically the feedback that we started getting was like people hated it and like it we just got so many support requests of people complaining that like they were entering 50 character along passwords and they were basically writing whole novels into the password passphrase field and um and they were still not hitting the entropy requirements and it was and we were seeing also a lot of lost passphrases and people emailing support being like i'm locked out of my account and that's being like you know we can't

help you sorry um so that was pretty bad i mean what we found is that people are basically mad bad at making passphrases for many of the same reasons as passwords um i mean on one side they're just not used to it and so these rules like were confusing and had some overhead but the other thing is that they're still predictable like when we get a request that verbatim said well i've entered 50 characters and i'm not passing this and we're like well i mean we have this tool and it's like yeah if you enter a password password password password you know it might be 50 characters but it's not going to fly um the other problem we had is that you

know some people succeeded and then we were talking to our users to get some feedback and one of the things that constantly came up was like oh yeah i just use like this like this quote from like my favorite book or my favorite band which also happens to be listed as one of my likes on facebook or tumblr or whatever other platform that's really easy for people to find in these cases you can see that these drake and bon jovi quotes actually passed this 100 bit standard another problem is that language is pretty predictable like you know we like to think we are able to create random statements or sentences but as it ha you

know as is the case language follows patterns and machines are really good at identifying this this is uh some data from google's engram charts and basically what it's saying is that if you enter the word you what words are most likely to follow because it's not going to be every word in the english dictionary it's going to be you can you are and it can actually go to multiple levels like you cannot you can get you can see and you can see how this really quickly narrows down the you know possible uh uh options that a machine needs to run through another issue is that when you let users pick their own um their own passwords

their memory is fallible like uh i would say close to half if not more of our forgotten passphrase support tickets were resolved by did you capitalize the first letter did you add a period at the end and that resolved many many many many the other fun part with this user uh you know user chosen uh system with meters was how meters themselves work in this case it's like cool you did it like okay cool this is my super sweet password i'm gonna just put in this last letter and uh too bad like password doesn't isn't appreciated by xbin or dropbox so it's just like really frustrating experience to be typing and being like yeah and

then never mind so you know we're still sort of at the same question of like how how can we actually ensure that um you know users meet this standard in this case passive aggressive password meter had some great advice you know maybe if you just mash your head onto the keyboard it would be more secure which could also mean maybe if you just had a way to randomly generate content then you know users wouldn't have to think about the passwords they create and we could get reliable standards for estimating password entropy the best part of all this is is that academics have actually studied this already and found that it doesn't matter if users pick their own passwords or they're

given to them they are just as likely to remember or forget them either way so it's like well okay if it doesn't make a difference why not let it be random oh um so then we get into i guess some of the problems with past phrases um i assume most people in this talk are familiar with this comic this comic is you know five years old now and this is what basically got me interested in past phrases in the first place i didn't know what they were like a year and a half ago and i saw this and i was like oh wow this is awesome this is a really easy solution to avoid things like this that i get asked all

the time this is a particularly bad list this is a real list of password requirements if you can't see it says you need exactly eight characters you need at least one letter one number and one special character but those special characters can only be at hashtag or dollar sign um it can't be in the first or last it has to be like a different one than you use over the past five times it's huge and terrible it requires a simile a metaphor and the name of one of your children um and i mean it could be no it doesn't and i mean it could be that pass phrases are also a fair bit longer right like on

the left is a randomly generated password and on the right is a passphrase and you can see the passphrase has almost twice as many characters on the other hand the pass phrase only has five chunks of data that you need to remember with the 13 character password i have to remember 13 characters with the passphrase i remember five words another issue is that passphrases need to be localized you can't just give an english passphrase to a french speaker and expect them to remember these five english words and that means you also need different dictionaries for every single language and it also means each language presents its own problems like for example german is not is probably going to have longer

past phrases than french or english um also against passphrases there's some evidence that passphrases aren't more usable than passwords one study found that five character passwords were significantly easier to remember and faster to type than equivalent strength three to four word pass phrases after um after using them for a few days however the same study found that six character passwords were significantly harder more annoying and less fun to learn than the five character passwords that you know good for us using larger dictionaries for passphrases didn't seem to affect memorability at all and that importantly in a word you rarely hear with pat with passwords and passphrases is that people found passphrases more fun to learn

on the other hand when we started looking at what happens when you use stronger requirements for password passwords and passphrases turns out that a six-word passphrase is significantly more likely to be memorized faster and more reliably than a 12 character password when using a service over a couple weeks so okay so what makes what are the could it be that the problem with passphrases is actually just the words so our question was what actually makes a good passphrase dictionary and and that's something that we hadn't seen a lot of research on so so what we looked at was you know the most famous word list is the dice where list and so the idea there was uh almost 8

000 words uh short words um and they're supposed to be common but arguably words like zlati wuhan and ncaa and boise as in boise idaho yeah um are not actually particularly common and also are very culturally specific and also there's a bunch of of you know like kind of gibberish that is included in there so and and one of the things that that for us was just not didn't quite go with our philosophy was the suggestion that uh you know if you don't know a word you should look it up and for us you know we want to reduce friction as much as possible for the user so we don't want people to like you know need to go to

some external site to look something up in order to be able to find it memorable so so that's something that didn't work for us so a different example is secure drop which uses a sort of trimmed down version of the diceware list and it removed a bunch of stuff from it like americanisms and symbols and gibberish but it still has some pretty difficult words um there's also the pgp word list which is very short and it also has very long and kind of hard to spell words it's actually meant for uh being spoken so that that factor's in there um there's also um so superior was originally based on on minilock and minilock came with a list of words which

was extremely long and um it uh we don't actually know where the source what the source from that is uh but it includes some pretty long and obscure words um for example does anyone can we have some more boos for anyone who can spell the word psoriasis without looking at your phone right now

i think yes yes congratulations okay what is an epic

which was used in the copernican system for describing uh you know the motion of the planets excellent congratulations we are all a bit small we have an educated audience yes we are all a bit smarter today so we also have up to the up to the minute up to the minute which is minute um so this is fun um we also have there's also this uh study uh correct horse battery staple which uses another uh word list which is from the uh contemp corpus of contemporary american english um and that's a much shorter word but it has a much shorter list but it does use really long words and it did actually find that character length was a

character length in a general passphrase or password was actually a huge factor in usability so the study found that five character passwords were more memorable than three to four um four word passphrases but length seems to have featured in that so that's so that's something that we took note of there's also this other study that we love um towards reliable storage of 56-bit secrets in human memory which uses a much shorter list as well um and it actually does do a lot of trimming um so it's chosen from google's engram corpus and it eliminates words that are close to each other um so hard to so easy to confuse and also removes prefixes so uh so that autocomplete can

work better um and excludes vulgar words plural words and slang so it's got a little bit of the sort of cultural element of trying to figure out what are good words for a human um so that's that's pretty great um but one of the things that we had a lot of questions about is what words are people actually likely to know and if they know them then we might extrapolate from there that those are the words that people are likely to remember and another question that we ask is what words are easy to type the other question for us given our entropy requirements is how many words do we want to use so these lists have a

huge range i mean from 200 and something to 56 000 so and a larger list means of course that we get more entropy per word but it also means um more characters per word on average and it also means that we get weirder words like epicycloid and whatnot um so we think that the problem in in all of these and all of these lists is that this is actually a ux problem and so a lot of these lists have been designed by developers and security professionals and they haven't been formulated from a sort of more um usability kind of side and so we that that's a problem that we found really interesting um and that is how we got to a cultural

approach to passphrases which we're also dubbing how seinfeld in game of thrones makes passphrases better so like a quick review some of the technical approaches is like well use shorter words use words based on patterns use phonetically distinct words or semantically distinct words make it friendly for autocomplete and autocorrect but some of the questions we were interested in is like well what words do people know what words do they know how to spell what languages do people speak what education level can we expect from people what words might offend people and like one simple thing is like well what do users actually think of this like are these words working for them or not you know talk talk to your end user

so you know on the first question it's like well what words do we pick and we could pick the most common words that appear in corpuses like from coca google engrams wiktionary and these are cool because they help us identify the most frequently used words in english but we're wondering if we should include all these words you know like these these lists include for example the harvard law review which has a flight grade reading level index saying that you should have been in school for about i think 33 years to understand most of the content in there in contrast dr seuss has a great index of one i.e you can have almost no education and be able to read uh

some dr seuss books so basically what we think is that well what if we sourced the most commonly used words from the most accessible sources because we know language has some social biases of like who gets published and where and in which mediums and the mediums themselves have different words like the most common words appearing in printed media is not the same as what appears online is not the same as television and film and so you know when trying to source words to protect the kingdom uh you know we are we used popular film and television um for most languages corpuses from opensubtitles.org and we did this on one side because you know popular video was accessible and

well-known tends to get translated on its own anyway or also you know if you're looking at other languages there's also local media that gets closed captioned and all of that also ends up on opensubtitles.org yeah opensubtitles.org actually as a source is fantastic because in many cases it's also local translators too and so people from the from that language will actually be the ones um you know recreating their phrases in a more culturally relevant way but importantly subtitles were actually studied and found to have a you know that the word frequency data from subtitles is more accurate this is a more accurate indicator of reading performance in a culture and word recognition that when uh studying french

american english speaking dutch chinese and greek youth they actually found that the way kids were learning was most accurately measured by the words they knew in subtitles not by corpuses from written texts so if we you know take that as our source we still have the question of like well how many words you know like do we keep it small do we keep it huge um do we take you know the common wisdom or you know do we just ask the question well how many words is someone actually likely to know um for this we used uh a website called testyourvocab.com it's basically been a long-term research project that i think has been on for

like 10 years where they just asked people to take this quiz online and it'll measure their vocab and we wanted to try and take one of the lower percentiles because we wanted to be as accessible as possible which for an 18 year old so you know high school graduate possibly in the lower 20th percentile it was measured that they would know about fifteen thousand eight hundred words so using so we just meal started but this is our guide so our process was basically find a large corpus of subtitles for the language you're working on compare that against the native language dictionary to try and get rid of slang sound cues or potential typos from the subtitles

organize the remaining list by word frequency isolate the 20 to 35 000 most common words um depending on the language and how large the corpus was remove offensive words from the list and then start trimming it down down by removing the longest words until the list is about 12 000 words long and the result is that we have 12 000 uh word lists of the shortest and most common words in a language with a cultural basis for assuming that people actually know what these words are it also means we end up with a larger dictionary with more entropy per word you can see that compared to one say a word list with only a thousand words you

would need five words instead of seven words in your passphrase for a similar strength it also tends to mean that when you have less words and they might be a little longer that it may not actually result in that much longer of a passphrase than just having more shorter words which has been some of the conventional wisdom for now so uh we threw this up here this is um from one of the newest eff short word lists coming up um that has an average of 4.5 characters per word and requires about six words to have similar strength to a five word um perio passphrase and they're on average they're very similar in length just so you guys can get an idea of what

this looks like we thought it'd be fun to generate the first uh like a few dice wear and perio pass phrases to see what they might look like and in general the diceware ones do are shorter um but we find the period ones to be a little more accessible or relatable i'm i'm personally a big fan of the first one that came up the reindeer ruling insanity rejoin reality um i i recognize most these words compared to things like you know 52 uw hera ncaa did make an appearance here for any basketball fans and idaho not boise though um yeah so this is just our visual comparison so um we haven't so i'll talk a little bit

about what's coming in the future in a bit but we had a chance to get our word list into a study done by at the university of bonn i don't think it's been published yet it should be out pretty soon um so it's um a researcher called sergey deshawn and some of his colleague colleagues and they were actually doing a study on public key uh comparison and recognition and so they threw our word list in in addition to hexadecimal base 32 uh purely numeric and the pgp word list so this is a bit it's it's not quite what we're doing i mean it's it's a public key um test not a not a memorization test uh for for past prices but it's

still fairly interesting because what we saw from that study is that um it was the fastest verification method among the the ones that they tried it was second best in terms of error detection so the best in terms of error detection was generated sentence like structures the second best it was also the second best in avoiding false positives interestingly the best in avoiding false positive was numeric and users definitely preferred it over non-language options so that's actually a pretty good mix given that it performs second to different things in different categories so of course that's not the same as seeing how memorizable it is but hopefully that's coming up um but but still this it was pretty a pretty cool opportunity

to get our to get our work into some some academic papers and we are looking forward to doing more of that so from what we have right now we do know that there are some pain points and so there are there are some things that we want to take a closer look at and we'd love to talk to uh to participants about um so one of the things that that we will ask ourselves is you know do subtitles from friends and grey's anatomy belong in you know various other dictionaries um we do definitely counter balance because we check uh the the words gener generated by the subtitles against the dictionary in the language that we're that we're using um but

there's still you can really see the influence of like british colonization and american culture um yes being norwegian i can tell you right now that the phrase how you doing is something that anyone in norway who has seen friends they are using that phrase if you did it in norwegian nobody would have any what you're talking about at all good to know so so yeah so so there's some tricky things there uh though like you know like so something might be more recognizable in one language and not translate very well um so so there's some trickiness around there um we do see that a lot of names even english names end up in the corpuses of every single language

um and that that that's you know that's a bit of a question mark for us so for example we ask ourselves if we should strip those names out and should we strip proper nouns because they're not a good point of reference for for a lot of people but then on the on the other hand um i think this is a particularly cool passphrase elizabeth crowned and human glamour typhoon and sometimes if you have a name in there that's actually i feel like that might have some points for memorability um especially if you know someone by that name um and i mean some of this might be nitpicking but we're trying to make something that that makes people have

a very fluid and pleasant experience and so little cultural things are important to take into consideration i guess

i didn't mean for it to excellent some names will translate differently for example pairs would be in english per se yeah it's it's pronounced differently and it means something different my name also um so um i don't know how you factor that in yeah totally that's it's it's super tricky actually um subtitles are interesting for that because subtitles aren't isn't the same corpus depending on the language like the media we showed is all anglo media but when you look at the subtitles.org you'll see tons of material from like local news or documentaries or material that may have never been in english either so so it actually will frequently represent the local culture you'll see more of the local city names of the

names of people in that uh language or place that can have its own problems like you know when you get to the point of comparing like brazilian portuguese versus uh portuguese portuguese but um you know things we paint like like we said pain points so other stuff that we haven't quite dealt with is uh words that are hard to spell occasionally we manually see something that's like oh my god this is a this might be a common word but people spell it wrong all the time like how many people spelled definitely wrong lots of people spelled definitely wrong so it's the kind of thing that that you that we might want to like just remove

that word because we know sort of anecdotally that it's a tricky word to spell um the other thing is words with short edit distances homophones that's something that we don't have a systematic way of dealing with particularly not across languages and we do so we're a pretty multilingual team we're based out of montreal and so everything isn't always in english and french and my native language is spanish and we've got some russian speakers on the team and a turkish speaker and we've got we've got a bit of a range but still we're branching out a lot into languages where we don't have someone who can who can obviously who can notice the obvious flags and one of the flags that we do have so

far is actually grammar which is and there's some stuff that in english isn't as obvious but once you get into most other languages things like this where you might have so the first spanish example uh there's a noun that is masculine and plural followed by an adjective that is feminine and singular so that actually causes a lot of cognitive dissonance and i mean we don't have any study to base this on but we've it's it's fairly jarring to see and probably that affects uh at least the user experience if not the memorizability because you would probably tend to make them agree when you're typing it out um so that's uh and the same thing happens uh in french that's an example

where the adjective is um feminine and singular and um and the uh the noun after it is plural so that's something that we might want to look at you know like just taking roots of words and um and making them agree um and then that actually can also help a lot in terms of edit distance so if anyone here speaks french french has this thing where there's a lot of forms that are spelled differently but they're pronounced the same and even native speakers actually confuse them a ton so if we could actually just unify that and make sure that we're only using one form of a verb that sounds and it sounds a particular way um that would probably go a long way

to uh fixing people messing that up um so so yeah so and then when we look at grammar i mean each language has particular things that we could probably do and it's a gigantic rabbit hole to go down um so but that's that's something that we're super interested in so if there are any linguists in the room we'd uh we'd absolutely love to talk to you um so um yeah there's some other cool stuff it's recent so this actually came out like uh a moment of panic when like two weeks before we were supposed to give this talk uh joveno released a new dice rare word list that uh is phenomenal we're really interested in playing around with this soon but um

basically they removed a lot of the weird words from dice ware they did something similar they sourced familiar words through ghent university's center for reading research removed difficult to spell words and homophones remove vulgar and offensive words uh and even did i think this is fantastic is identified uh the concreteness of words and what they mean by that uh is for example you know picture a screwdriver you know it's not hard you see the handle a little metal bit maybe star shape or whatever now now picture love it's a lot easier in the xkcd style to picture concrete objects if you were told to sit down here's five words that you can picture now try to picture them

like that comic with a little horse and a battery and a staple all together um it's a lot easier than trying to picture what love looks like so there um but actually made three lists one is um the initial dice wear length uh one is a reduced one that only requires four dice instead to hopefully speed up the process of creation it includes uh the most common words and then another one that's actually designed uh to be tuned to auto correct and auto complete which could be pretty fantastic on mobile in the sense that you would only need to ever type three characters to get your word automatically filled in each time so for like a visual comparison you know

we put them up to see how uh these uh our wordless and the short and long ones might compare um one thing to note on the short one is that if autocomplete and autocorrect turned out to work well people weren't over typing every time when they were trying to enter the words they would only actually be three characters per word um on the other hand i feel like password fields don't usually allow autocomplete so that's a kind of another set of challenges there's definitely some implementation challenges for auto correct and auto complete here um the other thing which made rounds in like popular you know tech blogs and things like that for a bit was research

on passphrase poems and someone uh this was i oh my god i'm i'm embarrassed to say this name gaza vinijad and knight in their essay how to memorize a random 60-bit string tried comparing sentence formulations the xkcd method and then poems because they were like you know the ancient greeks are able to like memorize a thousand page epic poem maybe we could do this to memorize a few passwords and what they found was uh impressive so these were the recall rates for their different their different methods and the user preference and the poetry method like vastly outperformed the two sentence methods and even the xkcd method and importantly like people really liked the poetry method uh versus the xkcd1 users

you know again when we set minimum standards you'll notice that users actually preferred the ones that they got wrong um so it's one of those things where maybe the user doesn't always know best yeah it's probably one of those things where they insert the correct grammatical forms rather than the incorrect ones there's two questions maybe over there are first in like thousands or people no this is people this is a pretty small like hands-on this is like a very significantly insignificant they're uh yeah it's a small study for sure the other question would be um with this the user preference was that control for in other words were those users possibly thinking they preferred those because they would be

more secure as opposed to the other ones i think they were measuring frustration but i don't remember exactly i'd be interested what control was used for that just to make sure that it wasn't based off of the user assumption that they were looking for a secure password as opposed to

so yeah no but we we can also give you that stuff that study if you want to see it i'm sure we have it oh actually it's also cited excited yeah um so some of the future work that we have in our in our development pipeline is some little things that we can tweak in terms of the user interface and user experience in general um so we want to kind of provide some sort of incentive uh for for people to do space repetition um so you know i don't know if anyone has used authy authy prompts you um to to enter your passphrase every once in a while to make sure you didn't forget it

um but that's honestly a little bit infuriating so we figure we can maybe add some spice to it by at minimum showing like a cute animation when you do it and maybe offering some rewards uh like more storage space on our on our platform um we can also do some stuff around trimming ignoring and inserting white space for example so that if you get if you get the spaces wrong uh we correct for that um then looking at the autocorrect and autocomplete stuff that justifyno's been doing that that could be really interesting so we might we might dip dip our our toes into that um and then another question that we just mostly are confused by is is how to

deal with special characters um because you know you can you know we have a german dictionary but if a german speaker has a german passphrase that has a bunch of umlauts and maybe a nest set in it then like if they travel and they're accessing uh their their account from someone else's computer that doesn't have these characters then they might be a bit screwed so there's a couple of different ways that we could deal with that that we should um that we should take a look at um but that's that's definitely something that's that's in our future the other thing is whether it makes sense to let users choose some things so that can be like usually we have just

a button that says regenerate passphrase we generate passphrase and it just cycles through a bunch of options but we could allow people to just regenerate individual words instead of the whole thing at once uh so if you know if you've got something that you like but the last word is something that you think you can't spell then you can just reroll that one um or the other thing is allowing uh insertion of additional words in between the ones that were randomly generated so for example like an article or a preposition to make the sentence a bit more sensical to you that could be a thing that could work um and also just you know allowing

insertion or edits to make something plural to add a suffix whatever um that's uh that could be interesting as well um and and we would still have the sort of guaranteed minimum amount of entropy from the the five or more words that are generated by the system so so that could be cool um the the next thing that that's coming up as well is we're currently designing a field study with the folks at the university university of bonn um and so what they have they have a bunch of different ways of generating uh passphrases and some control encoding and and whatnot and so what we're going to do is we're working on a research edition of perio where users will be

able to use their regular account but they will kind of go through a wizard that tells them okay you are participating in a study um we're going to measure the following things um and you will you know you will have your you will be using your passphrase encoded in a different way using a different generator so so that's we're going to test for for memorability for speed of entry for accuracy and for user preference so right now what we're getting going through is just kind of designing that study because it's fairly tricky to to figure out how to control for certain things um and and then in general you know we're very interested in doing field studies

and working with academic institutions but i don't want to interrupt i was going to wait until you're ready to switch lines all right i i have i've switched slides okay you said but you were continuing did you make an outrageous speaker request today yeah i think you did to ask for cats you asked for kittens oh yeah that's that's amazing sweet on twitter that is like the best thing that has ever happened to me and it wasn't even me it was you this is why it never hurts to ask all right i i have learned another thing today not just what an epicycloid is um so as i was saying in terms of designing um

in designing studies i mean we are a security secure communications tool so it's very tricky and we don't want to turn this into this kind of like horrible thing that people don't read where you're like clicking through a bunch of like text so there's a lot of a lot to be done in terms of designing transparent studies and having people actually understand what it is that they might be able to opt into so that's uh that's a really tricky question and one that we'd also love to talk to folks about um and on that note um there's a lot of ways that we would love uh for you guys to help out um so we would love to have native

speakers review our dictionaries so i had a it's it really helps uh for example i happened to luckily speak german but i hadn't caught this until we were preparing for this talk that actually when we were generating our german list we left out nouns entirely because because they're capitalized yeah and and we and our script just didn't account for that and um yeah and luckily i was like i was like regenerating in german and was like there are no nouns what the hell and then it occurred to me why and so i need to go regenerate that um so the other thing is like you use our dictionaries they're freely available um and just use our passphrases if you want

we have a standalone passphrase generator uh passphrases.perio.com where you can just roll in any of the languages um you can also come to us with any research that we've missed or any research that you want to do um and we're also we would love to collaborate with more folks from uh from the research community um and uh yeah and we'd love to work on field tests specifically um if folks are are interested in that so you can contact us through twitter and you can check out our the passphrases site you can check out the app as well and you can check out our code on github the app is itself open source so uh yeah that's i think that's all we

have for you when we say research i really want to stress that we don't just mean security researchers if you're into like ux usability design like we we feel like this is a really crucial and often um uh underappreciated and underutilized skill set within the security community so even if it's like what colors you think the passphrases should be when they're presented to users to help memorability like we want to talk about that yeah absolutely all right thank you yeah

they come out so before i open for questions i'm just going to say that skylar you told me and you really revealed yourself you didn't know what pass races was like one and a half years ago that's true and flow you say you're not an information security research or you consider yourself to be just a developer i'm a developer for sure yeah i am absolutely amazed about this talk so i i will actually you know one more round of applause first this is really good and now for now for questions raise your hands oh jeff wants to ask a question well i'm actually going to bypass jeff first too much enthusiasm there you go uh it's a it's a

pretty uh maybe obvious uh question um have you considered since you even named them sentences have you considered making the phrases that are longer than a couple of words maybe more syntactically arranged to improve memorability so that was actually one of our first thoughts in that last study we uh you know briefly went over the poetry one actually found that they tended to not work and the problem with it was that um on one side they got longer when you to get randomly generated sentences it that also aren't you know easy to look up in a dictionary type attack or but of quotes um yeah you end up with fairly long sentences and users were really good at

remembering like the semantic meaning of the sentence but they weren't uh in fact we might actually still have it open but they weren't very good at remembering the specifics it would be like they uh where's this is reversed um yeah so we do have it here so they would remember things like those are some more examples of poems um but these are sentences right so it would be like they'd be assigned no dressing allowed under canon law in the youth group but the person would remember no dresses allowed and that this is actually what result was the biggest failure for the recall rate for these types of sentences which is why we were a little more interested

in the user introducing their um you know constructing their sentences because then it actually arguably would add more entropy right because then you can't just do a dictionary attack of like well we generated all these words but did the user add an article to it did they change you know a suffix uh i meant a little bit or less drastic uh sentence construction but i can talk to you later about this no yes that's the problem is so with our our standard was 67.5 bits these are only 60 bits i'm going to make sure that they will be at the party tonight so we can discuss everybody discuss with them all right so this is fantastic you've been struggling

and working out with the same problems that we've been thinking about with one password and trying to and and master passwords and by the way i'm a phd dropout in linguistics i actually have a bachelor's degree in linguistics but i don't talk about that okay um so you you know so you had estimated links um anyway i'll talk to you about what we've done about unicode and i had a whole bunch of different questions and points which have all gone out my head so i'll just mention one in the word list that i helped construct we deliberately chose to avoid derivational and inflectional morphology yeah um so that we didn't have a dresses dressing problem with

memorability um oh right and the one other question is where did you get your taboo words for non-english when you when you were excluding offensive words mostly from actual people there are there's there are some existing word lists it was this was like one of my more fun weeks at work where i was uh like my job is to find all the offensive words in all the languages possible and put them all in one giant nasty list of like 8 000 lines long and you you should come to norway because the higher the higher up north and now where you go the more common it is to actually use really seriously strong bad language as

part of your everyday talking to anyone you can't talk without it yeah yeah pretty much you know like if you come up there like from from oslo as an example you go far up north of norway then you know they will they will just look at you and feel sorry for you because you're not you know you're not capable of saying like every second sentence as an example we did so um one note we did have something interesting when trying to source this list um so i don't know if anyone here is familiar with violet blue um all right so her name shows up on a lot of offensive banned word lists and we and and someone who was trying to you

know go through our list to evaluate it like got really angry at us they're like what's so offensive about violet blue and we're like oh we'd like we will take that off there's nothing offensive to us but yeah there are um oh yeah she saw on twitter this was very public um one one other thing we did for the offensive master list is we actually took out words like nationalities or religions because that could that's better than taking out negative adjectives and we ended up with a lot of awkward past phrases with constructions like disgusting name of country and it's like okay well this isn't very nice yeah i don't ask very many questions but

because i'm a spelling dropout you mentioned difficulties what's wrong with leaving spelling difficulties in there if you really can spell them it pertains to you versus forcing me to spell something like somebody else spells it uh i don't know if i've described that correctly you mentioned you use the word difficulties somebody can't spell it possibly i can't spell it but i probably can but anyway why can't you use your own spelling difficulties i mean why are you restricted to to what you say you must use why couldn't a person actually use their own individual spelling difficulties even in relation to the words if i made myself and it yeah yeah if if you can misspell the

thing correctly why would you force me to have to remember something that i can't spell right anyway so so this is where um towards the end where we were saying it could be interesting to allow users to have some control so if the user is able to write their whole passphrase everything we said in the beginning it will probably apply because people are bad at making their passwords but if we say here's some random content and you can play around with it a bit like oh you know what i know i always spell definitely with this i instead or something like that then it could be possible that we could say well you're allowed to change

two or three characters in any word which if anything would

yeah it's but we're trying to we are deliberately trying to set a fairly low bar and and i don't think that any of the words

um but yeah but but i i see your point though in terms of spelling like especially if you consider british versus american spelling we don't want to force one spelling or another um but we no i mean we are right now yes we are definitely but we that's the kind of thing that we should consider allowing the user to modify that yeah all right fair enough yeah sure yeah so related to the earlier point about norway um have you considered the option of letting people choose to include vulgar words because yes that's more rememberable yeah uh no no no we've done that we've also like uh on the side for fun i have a simpsons dictionary i have a klingon

dictionary um klingon one's tough because apparently there's a lot of internal disagreements about how things are spelled and like formatted but like i was i was not ready for that that's another case where people should be able to pick their spelling yeah um there's been requests for elvis dictionaries there's like um yeah no there's there's a lot on the table for that i'm gonna leave it to arnold reynolds to ask you the final question first of all let me just reiterate that this is a really nice piece of work and i'm very impressed and if you guys generate a 776 word list i'd be happy to link it from the dice ver page if you like that

awesome um um and not to be defensive one little quibble is when you're comparing i mean a five word dice uh pass phrase has 65 uh 64.5 bits of entropy so yeah it's a little less than your target but comparing it to six is a little bit um pushing it yeah that's only true i think that was a mistake on ours because we were supposed to be comparing it with the 1296 dictionaries that just came out which would be um you know i i want you to know that i spent hours working on this but not weeks so so and and and one of the challenge and and i'll want to just be aware that

you're going to have to redo this regularly because the culture changes i mean i one of my dice rewards is a p and i don't know how many people in this room i actually looked that up like two weeks ago because i was like i was like yeah i'm sure you know but but you know that they've been out of business and i have some years that would be very familiar in the 1990s that are you know like would drop off the top of your tongue and they're like why are you why do you have 1987 in there right i mean so so there there is a there is a upgrade process involved here and the final thing i was going to

suggest is one of the things because one of these i've been on my little to-do list that i may never get to is you really don't want to generate actual sentences because they can be weak passwords because they may actually turn up in a search list for that were generated from songs or whatever and they're at least in english there's like a handful of words like the forms of the verb to be and a few pronouns that basically and and articles that if you remove them from your list you eliminate like 75 percent of possible sentences so that's something you might also want to think about um so two things one one i should say uh

yeah we were actually worried we're like we're like uh crap we're we like are we being too critical of diceware um because we uh you know again i started with this like a year and a half ago and i basically read everything on the diceware website um to get to get to here um as far as content one of the fun things about subtitles is that they actually get updated every year you know new seasons of tv shows come out and new shows come out entirely um so it does actually help uh as far as a source of language to stay somewhat relevant in fact i think you're even um you could if you wanted you went through

the effort actually narrow it down i only want to take subtitles from shows that have been released from between these years um so there's some fun things you can play around with subtitles as a source

yeah okay so again applause for period