Why OpenAI removed Web Browsing from ChatGPT

Thomas Hansen - Jul 7 '23 - - Dev Community

The same day we connected our ChatGPT chatbots to the internet, OpenAI disconnected ChatGPT from browsing the web. OpenAI's explanation was that, and I quote "their web browsing feature allowed for users to circumvent payment walls".

This can be difficult to understand for non-technical users of ChatGPT, so I will try to explain it as simple as I can. However to understand, we must look at how most media outlet payment walls works, and why they work like this.

How most payment walls work

Most payment walls aren't really payment walls, they're simple "HTML tricks" to hide the text for users not having paid. If you don't know what a payment wall is, here's an example from Wall Street Journal.

Wall Street Journal payment wall

Notice, I don't mean to pick on WSJ in particular here, the entire industry are using the same cheap click and bait tricks here, and due to traditional media desertification having happened the last couple of decades because of social media and the internet - You can kind of argue they're just doing what they have to do to survive.

However, the strategy they're using with these payment walls are unethical, immoral, and arguably in violation of Google's internally developed rules for SEO and content production. If the above company had any other name but "Wall Street Journal", they would have been blocked out of Google's search index, since it's a black hat SEO technique often referred to as "cloaking".

Cloaking implies showing one thing to the search engine, and another thing to the end user

The point is that the whole article actually is there, it is just "hidden" with HTML trickery ...

WHY?

The natural question then becomes why these companies are doing this. The reason is easily understood, they want to rank high for anything related to whatever the article's content is, while using the article in Google to "click bait" users into paying for a subscription based service. Like I said ...

This is unethical, immoral, and in violation of everything we find to be "decent human behavior"

When Google is scraping the above article, Google sees the whole article, and will use the content of the article as they're showing search results to their end users. When the end user clicks the article, he needs to pay to read it. This is flat out immoral business practice, and that OpenAI shuts off one of their primary features to accommodate for this, is quite frankly absurd. Paradoxically, we win as usual, so I shouldn't really complain, since we're getting tons of traffic from users looking for alternatives allowing them to connect ChatGPT to the web.

FYI - I couldn't imagine a gun big enough in the world to be able to coerce me into turning off this feature in our chatbots! If Wall Street Journal is of the opinion that what we're doing is "unethical", I will meet them in any court, any place on the planet, and defend my case - And highly likely win!

The alternative

The alternatives for Wall Street Journal, and other news media outlets, is to create a real payment wall, where parts of their content truly is behind a payment wall, and not just fancy HTML trickery - For then to produce high quality content, investigative journalism, such as the article you're reading now, to such attract readers into paying for more content. Today they're just an echo chamber, an extention of our governments, doing whatever they can to please those with powers, having failed their obligations towards society at large - Something clearly seen by how they've treated Julian Assange the last decade.

For you as a user there's also an alternative, our ChatGPT AI chatbots. These chatbots are connected to the internet and can search using DuckDuckGo. They will scrape whatever HTML they find, and use it in combination with ChatGPT to answer your question - Implying it will bypass WSJ's "cloaking trickery".

Conclusion

For ancient dinosaur companies and entities such as WSJ, New York Times, The Chronicle, and God knows who else to be able to dictate innovation, and the future of mankind, is simply absurd - And we will not have it. Either change your business practices, stay relevant, and survive - Or go die somewhere peacefully, like the dinosaurs did. You served us well for 250 years, but today you're prohibiting innovation, corrupt through to the bone, and there are no justifications for your existence what so ever. If you want me to prove it to you, I can prove it with two words ...

Julian Assange

Edit - We created an alternative ChatGPT search engine ourselves ... ;)

. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .