Web Crawler

  External URLs

Audit internal and external links for 404 errors and 301 redirects.

  Meta Data

Analyse page titles and meta descriptions and identify those that are too long, short, missing, or duplicated across your site. 

  No Cookies

An estimated 76% of people ignore cookies so why annoy lot of people? Say no to cookies and Google Analytics with a static site and server logs.

  Robots and Directives

Test which URLs are blocked by robots.txt, meta robots or X-Robots-Tag directives such as ‘noindex’ or ‘nofollow'.

Server WEb LOgs

Web Crawler, SEO Spider and Analytics

There is a reason why it's called Google Analytics and not Google Statistics. GA relies on client-side JavaScript which can be blocked therefore it is not a complete representation of the visitors to a site.

Server-side analytics have a several advantages compared to js tracking. There is no impact on your site speed and loading time, not blocked by privacy or adblocking browser extensions, includes visitors who disable JavaScript and no third-party integrations required.

Static Site SEO
web-crawler_medium.png

SEO Spider and Web Crawler

The SEO Spider is a website crawler to help improve your SEO by extracting data and auditing for common SEO problems.

Prevention is better than cure. Solve URL issues before they occur using the web crawler to check localhost before you deploy pages to a live site. Build your entire site and scan internal and external URLs to determine the status code and ensure you do not have 404 errors.

Checking external links to third party site is essential to SEO. Crawl a website instantly and find broken links (404s) and server errors. YouDoCMS web crawler connects the crawl results to the site URL database for internal and external URLs. 

  • Build and test locally.
  • Scan internal and external URLs.
  • Check and set robots index status.
  • Status codes and content type.
dashboard-sessions_medium.png

Analytics Dashboard

Accurate visitor statistics derived from the web logs that exclude known bots and user defined IP addresses. An alternative to Google Analytics that protects your data and your visitor's privacy.

Common visitor metrics such as browser type, OS, device, IP address and country.  Session data including page views, session duration and bounce rate. Create actions by URL for example downloads. Monitor performance with top entry and exit pages. 

  • Filter by browser, bot or status code.
  • Custom date range.
  • Session data.
  • Bounce rate.
  • Page performance.
  •  Process old log files.

WEB SERVER LOGS VS GOOGLE ANALYTICS

Server logs are more accurate.

Page Speed

 Speed is a known factor in SEO particularly for mobile. Use web logs to identify large pages which are slow to load to solve performance issues and improve your score on Google page insights.

100% Accurate

Web logs show everything, regular visitors, visitors that block Google Analytics tracking code, and traffic from non-visitors like robots and spiders. Therefore, web log analysis is more accurate than client-side tracking.

No Cookies

Say goodbye to cookies. No need for the annoying pop-up if you are not loading JavaScript tracking code which relies upon a cookie. Speed up your website and stop annoying your visitors by using Web server logs for analytics.

Missing Data

Ad blocking is growing and in the US 2019 it is estimated that 76m or 27% of users now block ads and tracking. If you want the complete picture use server logs for analytics.