Articles by Sean Murray

Banks and Retailers Are Secretly Evaluating Your Eyes

April Fools 2022

Last December, a leaked pitch deck from a boutique investment bank ended up in the hands of a veteran tech reporter for Scribner’s Gazette. Kirby Fitzpatrick, who spent much of the past year covering the Buy-Now Pay-Later beat, laughed when he saw that the package sent to him anonymously was titled “Eye-Now Pay-Later.” Fitzpatrick said he groaned when he read a note that was attached. “You have to see it to believe it,” it said.

Inundated with promotional press packets on a daily basis, Fitzpatrick said that the pitch was literally so corny that he was driven to take a look through it. He was glad that he did. The slides had a link to a demo on the staging server of a well-known national bank. The instructions were very simple, look into the camera of your computer and think about the happiest moment of your life for 10 seconds and then click “submit.” So he did.

The only thing that popped up on the next page was a number. It said 250.

“I had no idea what that was supposed to represent or mean,” Fitzpatrick said, “but the next slide said to go back and repeat the process but this time to think about the worst moment of your life.” Playing along, he clicked submit and was greeted by a new number, this time 5. It’s the next slide that opened his eyes to what the game was. The technology had somehow determined that his happy moment was worth more than his worst and the numbers represented the optimal price that a consumer should be presented for a fictional product where the retailer could still profit.

“You are more likely to pay a higher amount for something that will bring you extreme happiness,” the next slide said.

In the retail world, this is obvious information. The challenge, according to Fitzpatrick, is that big data knows what you clicked on and what you’re looking for, but it doesn’t have a great way to judge just how bad you want it and how happy it would make you. “You clicked on the dress, you compared prices on the dress, but a retailer has no way of listening into the debate playing out inside your head. They don’t know if it’s the dress you had always imagined yourself in or if you’re just looking for something to hang in the closet with no particular function in mind.”

Intrigued, Fitzpatrick called the bank that hosted the demo to inquire if such a concept had ever gone anywhere.

“By the next day,” Fitzpatrick complained, “I was told by my editor that I was suspended from my position and was locked out of my work accounts. The bank apparently accused me of a security breach and I was basically in trouble. I couldn’t believe it.”

But later that night his editor showed up at his house, knocking discreetly on his backdoor and holding his finger up to his lips when Fitzpatrick answered. The story, it seemed, was still alive, but had to be taken to someone bigger like the Wall Street Journal or the New York Times. The Gazette was out of its league.

The bigger papers, backed by powerful legal defense teams, assigned its top journalists to collaborate with Fitzpatrick. Within weeks, the collective determined that the technology was real and that it relied on measuring minuscule changes in pupil dilation to gauge happiness. It was all about the eyes.

“We thought the science was cool and we worried about how this could impact consumers if something like this were ever used in the future,” Fitzpatrick said. “We thought we had a cautionary consumer advocate story brewing but then we were shocked by what we learned next.”

The team found that a handful of mobile-first web-based retailers had already been using it and that the phone cameras of shoppers were being activated as they browsed these sites. The only way to see the price of an item was to click the picture of it and by then the technology had already analyzed their eyes to determine how much it was desired. The price they’d see on the next page reflected the retailer’s cost plus the analyzed margin of desire.

The team tried to game the system to see just how much they could make the prices change based on what they were thinking. The widest variation was achieved on a polyester scarf, which for one reporter was $39 and another was $799. “Nobody dared ask what kind of happy thoughts activated a price of 800 bucks,” the Gazette editor joked. “But I don’t think it was his ex-wife.”

Once again, the team thought it had its story wrapped up, until it discovered the technology was hiding in plain sight. Amidst all the html, javascript, and tracking cookies, was a tag embedded in the website [!slooflirpa] that activated the cameras and the technology behind it. The worry was that the team started finding the code all over the web, including on the websites of big box retailers and department store chains.

“We looked at the archived page of a retailer that had gone out of business in 2016 and saw that the code had been on its website even then,” Fitzpatrick said. “That shocked us to think that this could’ve been going on almost six years ago. It sounded impossible even. I started to think about everything I had overpaid for online over the years and shuddered at the ‘what if.'”

The editor of another large paper stunned the room into silence when he asked what year Apple had first released an iPhone with front-facing cameras. “We had to look it up and the answer was 2010,” Fitzpatrick said. “At that point, it was like ‘folks, what are we looking at here? Is this something that everyone already knows? Is this just how things work and I’m the only person who never knew?'”

The collective sought out help from an optical technology firm to get an opinion on their findings so far. Ekaf Eman, CEO of SEKOJ Vision, confirmed that not only was such pupil technology possible but that there had also been a huge rush for retailers to buy camera equipment that was capable of it.

“We did consulting for a lot of these stores,” Eman said, “and we assumed they wanted it to identify shoplifters and reduce theft, etc. but every time we brought up the technology in a security context, they were like ‘wait, security? oh right, yes, security, sure yes put it down as that.’ It was very suspicious but we didn’t think anything of it at the time.”

“I asked Mr. Eman if he worked on this before or during covid,” Fitzpatrick said. “And he says ‘covid? ha, oh long before… this was back in ’96 or ’97.'”

It was then that the collective took a step back, suspecting that it had possibly stumbled upon a national intelligence or homeland defense operation that it had inadvertently mistaken for something else. While awaiting to hear back from the FBI before proceeding further, the investment banker who authored the original pitch deck showed up at the office of Scribner’s Gazette. Having recently been let go, he laughed when he heard the reporters argue that the whole thing smelled like a CIA operation.

Kip, the banker’s preferred pseudonym, said that retailers had already moved far beyond price and were already using the tech to generate interest rate proposals on loans. “You ever hear of Buy-Now Pay-Later?” he asked. “That’s the latest frontier for eye analysis.”

Kip said that when he first started in the field, the technology was being used to gauge if an in-person consumer had looked at the price of an item on the shelf. And if not, it then assessed pupil dilation to determine how much it was desired. The desire was translated into a price that was then transmitted to the cash register they went to.

“I didn’t really believe it myself when I first learned about it,” said Kip, “but a friend of mine named Billy once partook in the intake of a certain substance and then decided he really needed to buy a pair of socks at the mall. So he gets to the register and the cashier, wide-eyed, tells him it’ll be $3,500. Billy just pulls out his Amex card and swipes it like it was nothing. Then he ripped open the pack and proceeded to put the new pair on his feet right there in the checkout lane. So I said, ‘damn, how are those socks Billy?!’ and he told me that they were freaking amazing.”

The tech company who built the algorithm reportedly contacted Kip afterwards to let him know that his friend Billy had actually reached a perfect score of happiness.

“This technology changes lives,” Kip said. “Billy told everyone he’s ever met about how he once walked around in a thirty-five-hundred-dollar pair of socks. He’s very proud of it. I didn’t have the heart to tell him how much I paid for the exact same pair.”

Fitzpatrick, of the Gazette, asked Kip how he approaches shopping knowing what he knows about the entire scheme.

“I only shop while mad and sad, just downright miserable,” Kip replied. “One time I cried until the same pair of socks was only 50 cents. Billy asked me how they were. I told him they were the worst, just the absolute worst.”