Bash: Display Web Page Content In Terminal

How can I fetch HTML web page content from bash and display on screen using shell utilities?

You can use any one of the following tool or combination of all of them to get the contents of a webpage in a shell:

[1] curl command – It is a tool to transfer data from or to a server using http/https/ftp and much more.
Tutorial details
Difficulty Easy (rss)
Root privileges No
Requirements curl/wgetlynx/w3m
Time 10 minute
[2] lynx command – It is a fully-featured World Wide Web (WWW) client/browser for users running terminals. [3] wget command – It is a free utility for non-interactive download of files from the Web. It supports HTTP, HTTPS, and FTP protocols, as well as retrieval through HTTP proxies. [4] w3m command – It is a text based Web browser and pager.

Installation

Above tools may not be installed on your Linux or Unix like operating systems.

Note: You need to login as root user to install required tools.

Debian / Ubuntu Linux install curl, wget, lynx, and w3m

Open a terminal and and then type:
$ sudo apt-get install curl wget lynx w3m

Fedora / RHEL / CentOS Linux install curl, wget, lynx, and w3m

Open a terminal and and then type:
$ sudo yum install curl wget lynx w3m

FreeBSD Unix install curl, wget, lynx, and w3m (binary package)

Open a terminal and and then type:
$ sudo pkg_add -v -r curl lynx w3m wget

Examples

You can use curl command to download the page:

curl http://www.cyberciti.biz/
curl http://www.cyberciti.biz/faq/bash-for-loop/

Use curl and store output into a variable as follows:

page="$(curl http://www.cyberciti.biz/)"
page="$(curl http://www.cyberciti.biz/faq/bash-for-loop/)"

To display content use echo or printf command as follows:

echo "$page"
printf "%s" $page

lynx command examples

Use the lynx command as follows:

lynx -dump www.cyberciti.biz
lynx -dump www.cyberciti.biz/faq/bash-for-loop/

The -dump option dumps the formatted output of the default document or those specified on the command line to standard output. Unlike interactive mode, all documents are processed.

wget command examples

The syntax is as follows:

wget -O - http://www.cyberciti.biz
wget -O - http://www.cyberciti.biz/faq/bash-for-loop/

OR use wget command to grab the page and store it into a variable called page:

page="$(wget -O - http://www.cyberciti.biz)"
## display page ##
echo "$page"
## or pass it to lynx / w3m ##
echo "$page" | w3m -dump -T text/html
echo "$page" | lynx -dump -stdin

w3m command examples

The syntax is as follows to dump web page content in terminal using the w3m command:

w3m -dump http://www.cyberciti.biz/
w3m -dump http://www.cyberciti.biz/faq/bash-for-loop/

OR use w3m command to grab the page and store it into a variable called page:

page="$(w3m -dump http://www.cyberciti.biz/)"
echo "$page"

Practical examples

Get the definition of linux from a dictionary:
$ curl dict://dict.org/d:linux
Sample outputs:

220 pan.alephnull.com dictd 1.12.0/rf on Linux 3.0.0-14-server  
250 ok
150 1 definitions retrieved
151 "linux" wn "WordNet (r) 3.0 (2006)"
Linux
    n 1: an open-source version of the UNIX operating system
.
250 ok [d/m/c = 1/0/30; 0.000r 0.000u 0.000s]
221 bye [d/m/c = 0/0/0; 0.000r 0.000u 0.000s]

Backup your del.icio.us bookmarks:
$ wget --user=Your-Username-Here --password=Your-Password-Here https://api.del.icio.us/v1/posts/all -O my-old-bookmarks.xml
$ more my-old-bookmarks.xml

Grab all .mp3s from url:

mp3=$(lynx -dump http://server1.cyberciti.biz/media/index.html  | grep 'http://' | awk '/mp3/{print $2}')
for i in $mp3
wget $i
done
See also
  • See man pages for more info – wget(1)
🐧 If you liked this page, please support my work on Patreon or with a donation.
🐧 Get the latest tutorials on SysAdmin, Linux/Unix, Open Source/DevOps topics:
CategoryList of Unix and Linux commands
File Managementcat
FirewallAlpine Awall CentOS 8 OpenSUSE RHEL 8 Ubuntu 16.04 Ubuntu 18.04 Ubuntu 20.04
Network Utilitiesdig host ip nmap
OpenVPNCentOS 7 CentOS 8 Debian 10 Debian 8/9 Ubuntu 18.04 Ubuntu 20.04
Package Managerapk apt
Processes Managementbg chroot cron disown fg jobs killall kill pidof pstree pwdx time
Searchinggrep whereis which
User Informationgroups id lastcomm last lid/libuser-lid logname members users whoami who w
WireGuard VPNAlpine CentOS 8 Debian 10 Firewall Ubuntu 20.04
8 comments… add one
  • Bob Aug 5, 2013 @ 14:53

    You can use elinks too; another text web browser.

  • Jalal Hajigholamali Aug 6, 2013 @ 5:02

    Hi,

    Thanks a lot

  • Curtis Aug 6, 2013 @ 6:14

    Lets not forget about plain ol’ links as well!

  • Mike Bravog Oct 20, 2013 @ 17:22

    As Bob said: elinks too. For me, the best option!

    For Debian/Ubuntu series users:

    # sudo apt-get install links

    That’s all.

  • rod May 9, 2015 @ 21:51

    Is there a way to display the page for humans, I mean, interpret in some way the markup language in the bash (with no image/css/etc obviously)?

  • siddharth das Jul 27, 2016 @ 16:10

    How many tab open in terminal

  • abhinav sarvari Aug 3, 2016 @ 8:00

    thank you mate

  • O Aug 21, 2016 @ 8:08

    I like elink very much as well.

Leave a Reply

Your email address will not be published. Required fields are marked *

Use HTML <pre>...</pre>, <code>...</code> and <kbd>...</kbd> for code samples.