Blue Crab - 4.9.8

Blue Crab is a versatile program that you use to copy the contents of a web site to your computer, in whole or in part.

• Conduct fast offline browsing and searching without an internet connection.
• Perform batch downloads of URL's
• Create a snapshot of a website for historical archiving.
• Collect specific types of resources such as images or email addresses.
• Search current content more thoroughly than a search engine right on your own computer.
• Check a site for broken links, or generate an HTML sitemap
• Create full page images of URL's (JPEG, BMP, TIFF, PNG, PSD, etc.)

With Blue Crab you can download all the content including HTML, PDF, graphics, video, file archives, etc., or use selective filtering to restrict downloads to specific kinds of files. For example, you can choose to save only the JPEG images Blue Crab finds, or just the PDF's.

Blue Crab has a special feature called the "Media Grabber" which you can use to easily download just the images (or movies) on a web site Moreover, you can view a slide show of the images as they are downloaded. You also have the option of "flattening" the images, i.e. putting them all into one folder, or preserving the folder structure on the server (just as when downloading a complete web site for offline viewing.)

Blue Crab is always gentle on the server, processing only one URL at a time. (You can even make it pause between successive downloads so as to not overwhelm the server.)

The demo version you download now is fully functionaly and runs for about ten days.

Major Features:

• Unique URL extraction algorithms yield very complete downloads compared to other products of this type.
• You can download web sites either for offline browsing, via re-linking of URL's, or to create virtual copies of them on your hard drive, i.e. backup a web site as-is.
• Supports both HTTP, HTTPS protocols as well as server realms. Detailed Grabber window shows server HTTP header response fields.
• Optionally display web pages as they are crawled using either your own web browser or Blue Crabs own web window.
• You can restrict a crawl to just the initial domain, or allow the program to "stray" into offsite URL's
• Blue Crab can remap "dynamic" URL's which contain path and search arguments so that resulting disk files are navigable offline.
• Image URL's into JPEG files (or any other graphic file type supported by Quicktime) to create complete pictures of web pages not possible simply by taking a single screen shot yourself.
• Blue Crab provides its URL imaging ability as a service menu item available from other applications.
• URL's can be imaged into files, or onto the clipboard.
• Built-in directory search displays found files as a hierarchical list, with double-clickable entries.
• A "batch window" enables you to download your own list of files. Simply drag and drop URL's into the batch window, or import them from an HTML file , or text file.
• Grabbing in "images-only" mode just downloads image files, optionally "flattening" the results (i.e. putting all images into one folder.)
• Reusable configuration settings control what is grabbed, saved or crawled:
• Filter by size, date, file type or even content
• Filter by filename, extension or path
• Bookmarks window for storing frequently crawled web sites
• Supports form submissions to enable you to begin a crawl from a login page.
• Supports cookies for more accurate downloads.
• Generate site maps that consist of a hierarchical display of URL's Such maps can be optionally filtered by extension.
• Find email addresses.
• Customizable user-agent (spoofing) enables you to download web sites which are platform specific.
• Optionally receive email notifications of Blue Crab's progress during a lengthy crawl.
• Dock badging provides visual feedback or progress when the program is in the background.