The Fed’s Analysis of Online Lender Satisfaction is Bogus

July 15, 2016 | By: Sean Murray

Federal Reserve

A strange statistic about borrowers and their supposed overwhelming dissatisfaction with online lenders is circulating where it shouldn’t be.

Only 15% of small business borrowers were satisfied with the loan they were approved for by an online lender, according to a Federal Reserve study published back in March. That’s actually not even what the study says but that’s the selective takeaway that some very influential people have gleaned from it. That figure in some way shape or form is being repeated at conferences, cited in academic papers, and even referenced in Congressional testimony. Apparently when it comes down to it, people are starting to believe that small businesses overwhelmingly hate their experience with online lenders because it’s something they think a very credible Fed study has confirmed.

It’s not a percent of satisfaction
The statistic in the study is actually representative of how many more people were satisfied than dissatisfied. That’s how they define it in their footnotes. 15% represents a net satisfaction score and it indicates that more borrowers were satisfied than dissatisfied. So 15% net satisfaction means that more than 50% of borrowers were satisfied.

Still though, banks scored higher than online lenders in the report so one might think that’s the important part at the end of the day. After all, this was a highly scientific study that we can still all make important decisions off of, right?

This wasn’t a scientific survey
The Fed disclaims any statistical meaning of their results in the fine print. Under methodology, it actually says:

The data are not a statistical representation of small businesses.
The SBCS is not a random sample of small employer firms, and therefore suffers from a greater set of biases than surveys that contact firms randomly.
Businesses are contacted by email through organizations that serve the small business community in participating Federal Reserve Districts.

The Fed’s own authors seem to be pretty clear in stating that the data is not random, it’s biased, it’s not statistically representative, and that it was collected by third parties who coordinated distribution of the surveys on their own accord. Therefore for a policymaker, this report has no scientific value. Nonetheless, the Fed opens the report by saying “Our hope is that this report contributes to policymakers’ and service providers’ understanding of the business conditions, credit needs, and borrowing experiences of small business owners.”

Admitted falsehood becomes truth
Even though the figures are meaningless, they are being recited over and over. A major US Treasury report published in May for example, cited the Fed’s 15% satisfaction figure in its own analysis of marketplace lenders.

Testimony by Greg Baer, the President of The Clearing House Association, cited the study as well during a Senate hearing three weeks ago, but seemingly interpreted it to mean that only 15% of small businesses were satisfied with online lenders. “As reported in a recent small businesses survey, borrowers are generally dissatisfied with online lenders,” he said.

Marcus Stanley, Policy Director for Americans for Financial Reform, made the same mistake when he testified before the House Small Business Committee last month. “The evidence indicates they often provide a substandard and even exploitative product – just 15% of small business borrowers from online lenders expressed satisfaction with their experience,” he said, while arguing why they should be regulated further.

An article published by Nerdwallet also made the wrong assumption. “In fact, only 15% of small-business borrowers in the Federal Reserve survey said they were satisfied with their experience with online lenders,” they wrote in a story back in April.

Sadly, these are just a few examples.

Now what?
Does a study that’s not random, biased, not statistically representative, and not even controlled by the researchers, do more harm than good for an industry? Perhaps, because readers assume that a government study is scientifically sound on its face. Despite the authors’ move to disclaim the scientific value of it altogether, a mischaracterization of what is even presented anyway is becoming an oft quoted truism.

What is happening now is that both advocates and critics of the industry are starting to believe that only 15% of small businesses are satisfied with online lenders. No study has ever come to that conclusion though, not even the Fed study. It’s all in the fine print that openly says its not statistically representative and biased. And if that wasn’t enough, they even add “caution should be taken when interpreting the results.” Which in other words means, don’t use this stuff for anything important.

Last modified: July 15, 2016