Yeah, sorry, I had to do some cleanup on my blog. I trashed about 90% of my old stuff. If you’re missing anything of value, hit me up, comment on this post or whatever and I might republish it. Thanks for understanding :)
THIS IS NOT MY PROJECT. I am just reporting on it. The creator is Chris Johnson aka @defaced!
I have been waiting to be allowed to talk about this literally for months. And the time has finally come.
May I introduce: Horseman.
Does this world need yet another crawler?
Yes. Yes, it does. We finally need a crawler that gives us more flexibility and ways to work with the data we find during a crawl. Simply exporting data using custom extractions sometimes doesn’t do the trick.
While my personal daily go-to crawling software is still ScreamingFrog, this new Crawler has found its way into my daily work as well. Let me quickly show you an example of what I wanted can do with Horseman, which would involve quite a few more steps if I wanted to do the same with any other crawler.
Why is Horseman getting me all excited?
Horseman is not like any other crawler you know.
Imagine you have the idea to check the sentiment of all the <h1> headings (or any [!] other element on the page) on your website.
My way of doing this right now would be to get a list of the elements I need. In ScreamingFrog, I already have the list of h1 elements. If I want to get any other element, I would need to set up a custom extraction and then export that list.
This CSV list / GoogleSheet data would then be fed into another software that I would need to find / write to get the sentiment of the specific text.
Custom Snippets FTW!
Let me show you the example I have started above – the sentiment analysis.
So within my custom snippet, I import the sentiment Node.js module and use its analyze method on the textContent of the h1 element and return the sentiment score.
This very simple snippet saves me a huge amount of time because I don’t have to build or use any other software than Horseman.
This way, you can easily extend the functionality of Horseman. It will do whatever you need it to do. Make a crawl as big or small as you need it to be.
Another example: Console errors
It doesn’t always have to be that bad, but it could potentially be. This is why Horseman has got a built in snippet for that.
Which would look like this in the crawl:
These are just two little examples of what Horseman can do for you. There are a ton of snippets already built into the app – and many more are to come.
Knowing our wonderful SEO community, there will be a huge “marketplace” of freely shared snippets in no time. In the future, I will publish my snippets on here as well, so anybody can use them.
tl;dr: Watch video, get involved.
Check out my Youtube video below. It will show you Horseman in action.
Where can I get Horseman?
At the moment, you need to become a Github sponsor to get into the early access program. Let me tell you: it is totally worth it. Chris Johnson aka @defaced is working on this thing and putting his heart into it. This crawler is really something that any SEO, especially the technical SEOs like me, need to be using to save time, effort and blood, sweat and tears.
Check it out, have fun with it and please support Chris by becoming a sponsor on Github.
Horseman is currently in its early access phase. I use it during my daily work. It’s a very, very promising new crawler that gives me the freedom to expand and customize it to get the stuff done that I need to get done.
There are some minor things that are still in the Guthub issue backlog (and which you can also see in the video), but this is mostly polish / design work. Horseman is already a great tool and a much appreciated addition to my SEO toolkit.
aka: the #titlegeddon bookmarklet
Google has stirred up some dust all around the SEO world with their rewriting of websites’ titles.
SEO folks are now trying to figure out how Google is rewriting their titles.
Check out how simple it is to see all X-Robots-Tag response headers in your Chrome DevTool’s “Network” Tab.Add X-Robots-Tag response header to Google Chrome DevTools
Es war endlich soweit. Am 15.05.2019 bin ich nach 16 Jahren zum ersten Mal wieder in den Libanon geflogen. Es ging mir bei dieser Reise darum, die alten Orte, die in meiner Erinnerung eine große Rolle spielen, wieder zu besuchen.
Meine Reise in den Libanon war nur sehr kurz – ich hatte nur zwei komplette Tage und ein paar Stunden am Anreisetag zur Verfügung, so dass ich alle Orte, die ich besuchen wollte, bereits Wochen – oder sogar Monate – zuvor komplett durchgeplant hatte.
Dieser kleine Blogbeitrag soll in erster Linie dazu dienen, meine Erinnerungen für mich selbst festzuhalten. Wenn er darüber hinaus noch jemandem gefällt oder das Bild des Libanon im Kopf des Lesers etwas in ein positives Licht rückt – umso besser.Continue reading “Libanon 2019 – Erinnerungen”