Newton's third law, expressed as violence.
I'm on one of my irregular "if you put trackers into a smart phone app, you should masticate a toaster" kicks, which means I'm scouring F-Droid for new apps/games. I found Forkyz, and even though ya boi has never been much of a crossword fanatic, I figured I'd give it a whirl.
I opened the app, hit the plus button, and downloaded all of todays puzzles. Downloading the default sources includes the USA TODAY crossword and, since I have a high neanderthal DNA content, figured that would be a great start. That was the correct choice, as it was a good mix of pop culture and je ne sais quoi.
A couple days later, I opened the app and refreshed all day but could never get the USA TODAY to download. A SearXNG search led me to this Reddit post, so I figured "let's (w)get this bitch."
[09:47:48] user@server:~$ wget http://picayune.uclick.com/comics/usaon/data/usaon220627-data.xml
--2022-07-28 09:47:56-- http://picayune.uclick.com/comics/usaon/data/usaon220627-data.xml
Resolving picayune.uclick.com (picayune.uclick.com)... 66.6.101.188
Connecting to picayune.uclick.com (picayune.uclick.com)|66.6.101.188|:80... connected.
HTTP request sent, awaiting response... 200 OK
Length: 5166 (5.0K) [text/xml]
Saving to: ‘usaon220627-data.xml’
usaon220627-data.xml 100%[=============================>] 5.04K --.-KB/s in 0s
2022-07-28 09:47:56 (687 MB/s) - ‘usaon220627-data.xml’ saved [5166/5166]
Let's open that file in Forkyz.
Now, having to:
· SSH into my server
· cd to a specific directory
· Ctrl + R and comb through all of my wget commands
· Change the date in the argument
every time refreshing doesn't work sounds mighty annoying. Let's script it.
mkdir /mnt/pool/crosswords
cd /mnt/pool/crosswords
nano get_usa_today.sh
Here's what we'll put in ./get_usa_today.sh
#!/bin/bash -x
cd /mnt/pool/crosswords/
wget http://picayune.uclick.com/comics/usaon/data/usaon$(date +"%y%m%d")-data.xml
Finally, automate the thing.
pico /etc/crontab
...
30 9,13 * * * user /mnt/pool/crosswords/get_usa_today.sh
A few days later, we now have a month's worth of USA TODAY crosswords.
[10:14:30] user@server:~$ ls -lat /mnt/pool/crosswords/
total 388
drwxrwxr-x 2 thomcat thomcat 4096 Jul 28 10:06 .
drwxrwx--- 15 thomcat thomcat 4096 Jul 28 03:56 ..
-rw-rw-r-- 1 user user 5593 Jul 27 18:59 usaon220728-data.xml
-rw-rw-r-- 1 user user 5593 Jul 27 18:59 usaon220728-data.xml.1
-rw-rw-r-- 1 user user 5469 Jul 26 18:59 usaon220727-data.xml
-rw-rw-r-- 1 user user 5469 Jul 26 18:58 usaon220727-data.xml.1
-rw-rw-r-- 1 user user 5413 Jul 25 18:59 usaon220726-data.xml.1
-rw-rw-r-- 1 user user 5413 Jul 25 18:58 usaon220726-data.xml
-rw-rw-r-- 1 user user 5362 Jul 24 18:59 usaon220725-data.xml.1
-rw-rw-r-- 1 user user 5362 Jul 24 18:59 usaon220725-data.xml
-rw-rw-r-- 1 user user 5511 Jul 23 18:58 usaon220724-data.xml
-rw-rw-r-- 1 user user 5511 Jul 23 18:58 usaon220724-data.xml.1
-rw-rw-r-- 1 user user 5474 Jul 22 19:00 usaon220723-data.xml.1
-rw-rw-r-- 1 user user 5474 Jul 22 18:59 usaon220723-data.xml
-rw-rw-r-- 1 user user 5231 Jul 21 18:58 usaon220722-data.xml.2
-rw-rw-r-- 1 user user 5231 Jul 21 18:58 usaon220722-data.xml
-rw-rw-r-- 1 user user 5231 Jul 21 18:58 usaon220722-data.xml.1
-rw-rw-r-- 1 user user 5434 Jul 20 18:59 usaon220721-data.xml
-rw-rw-r-- 1 user user 5434 Jul 20 18:59 usaon220721-data.xml.1
-rw-rw-r-- 1 user user 5383 Jul 19 18:58 usaon220720-data.xml.1
-rw-rw-r-- 1 user user 5383 Jul 19 18:58 usaon220720-data.xml
-rw-rw-r-- 1 user user 5463 Jul 18 18:58 usaon220719-data.xml.1
-rw-rw-r-- 1 user user 5463 Jul 18 18:58 usaon220719-data.xml
-rw-rw-r-- 1 user user 5432 Jul 17 18:58 usaon220716-data.xml
-rw-rw-r-- 1 user user 5474 Jul 17 18:58 usaon220718-data.xml.1
-rw-rw-r-- 1 user user 5474 Jul 17 18:58 usaon220718-data.xml
-rw-rw-r-- 1 user user 5474 Jul 17 18:58 usaon220718-data.xml.2
-rwxrwxr-x 1 user user 122 Jul 17 11:36 get_usa_today.sh
-rw-rw-r-- 1 user user 5439 Jul 16 18:59 usaon220717-data.xml
-rw-rw-r-- 1 user user 5488 Jul 16 18:59 usaon220715-data.xml
-rw-rw-r-- 1 user user 5439 Jul 16 18:58 usaon220717-data.xml.2
-rw-rw-r-- 1 user user 5439 Jul 16 18:58 usaon220717-data.xml.1
-rw-rw-r-- 1 user user 5196 Jul 15 18:59 usaon220714-data.xml
-rw-rw-r-- 1 user user 5505 Jul 14 18:58 usaon220713-data.xml
-rw-rw-r-- 1 user user 4721 Jul 13 18:59 usaon220712-data.xml
-rw-rw-r-- 1 user user 5464 Jul 12 18:58 usaon220711-data.xml
-rw-rw-r-- 1 user user 5438 Jul 11 18:58 usaon220710-data.xml
-rw-rw-r-- 1 user user 5459 Jul 10 18:59 usaon220709-data.xml
-rw-rw-r-- 1 user user 5487 Jul 9 18:58 usaon220708-data.xml
-rw-rw-r-- 1 user user 5453 Jul 8 18:58 usaon220707-data.xml
-rw-rw-r-- 1 user user 5448 Jul 7 18:59 usaon220706-data.xml
-rw-rw-r-- 1 user user 5503 Jul 6 18:58 usaon220705-data.xml
-rw-rw-r-- 1 user user 5368 Jul 5 18:59 usaon220704-data.xml
-rw-rw-r-- 1 user user 5400 Jul 4 18:58 usaon220703-data.xml
-rw-rw-r-- 1 user user 5586 Jul 3 18:59 usaon220702-data.xml
-rw-rw-r-- 1 user user 5442 Jul 2 18:58 usaon220701-data.xml
-rw-rw-r-- 1 user user 5062 Jul 1 19:00 usaon220630-data.xml
-rw-rw-r-- 1 user user 5462 Jun 30 18:58 usaon220629-data.xml
-rw-rw-r-- 1 user user 5552 Jun 29 18:58 usaon220628-data.xml
-rw-rw-r-- 1 user user 5166 Jun 28 18:59 usaon220627-data.xml
Some notes on the above:
· All of the crosswords pre-July 17th were wget'ed manually, one-by-one. Haven't yet figured out the regex that will allow me to download a bunch at once.
· The ls -la date appears to be the timestamp the xml was created by USA TODAY and not the date I grabbed the file. I'm guessing that's expected, and shows wget ignorance instead of bad commands.
· I should really learn wget / curl