Showing posts with label SDR. Show all posts
Showing posts with label SDR. Show all posts

Friday, October 3, 2025

How to download WSPR and FT8 logs from a WEB-888 SDR

Since firmware version v2025.0119 the Web888 is able to log FT8/4 and WSPR decode information to the messages file of the OS. Because a reboot or a simulated reboot, i.e. a power failure will remove any log file from the Web888, I use the following to store the log onto a different host. Additionally I'm also able to further process the log, but that's a different story.
I use a Raspberry Pi (RaPi) as my storage system however, any *nix type system, including the "Fruity MAC" or the "Shattered Windows" (with the *nix add on) could be used by following the below steps. 

Note: Many ways lead to Rome as they say, and this is only one of the many ways this can be achieved, YMMV!

Prerequisite:

  • A little knowhow of how to abuse a keyboard.
  • A little knowledge about some form of *nix (MAC-OS, Linux, etc.).
  • A Terminal program (Putty, MobaXterm, xterm).
  • POSIX *nix commands.
  • ssh
  • ssh-keygen
  • the IP-Address of your Web888.

The step we will take to create the system,
    1. Create Password less access.
    2. create a key
    3. copy key to Web888
    4. modify Web888 boot image
    5. create file store
    6. test system
    7. automate system
Note: I will use the following syntax throughout to help distinguish between commands and command echos.

This is a command you type at (into) the terminal, you can also copy and paste.

And the below is a command echo,

Ok, well done grasshopper.

Create Key

Let's log in to our *nix system with our prefered Terminal application.
Since we are connecting to the Web888 via ssh will will create a set of ssh-keys to enables us to do so.The ssh-keygen application is the tool for us to set this up.

/usr/bin/ssh-keygen -t rsa -b 4096 -C "VK5HW"

Let me quickly explain, the -t specifies what type of key to create, the -b means how big/long the key should be and the -C is a comment field. You can use the comment to add your own callsign or any other comment you wish to the key.
After you pressed the [ENTER] key, you will be greeted with the following text:
Generating public/private rsa key pair.
(Patience is a virtue) you need to wait a little before you'll see  
Enter file in which to save the key (/home/hw/.ssh/id_rsa):
We will accept the default (/home/hw/.ssh/id_rsa) and press the [ENTER] key.
Now the next is the secret ingredient for password less access.
Enter passphrase (empty for no passphrase):
Yep, you guessed it, we hit [ENTER] as we do not want a passphrase (password)! And so the next question is answered exactly the same!
Enter same passphrase again:
Hit that [ENTER] key again.
Your identification has been saved in /home/hw/.ssh/id_rsa
Your public key has been saved in /home/hw/.ssh/id_rsa.pub
The key fingerprint is:
SHA256:some blah here VK5HW
The key's randomart image is:
+---[RSA 4096]----+
| some blah here  |
+----[SHA256]-----+

Let's quickly check,
ls  ~/.ssh/*rsa*
/home/hw/.ssh/id_rsa  /home/hw/.ssh/id_rsa.pub

Two files have been created and we need to install the id_rsa.pub, which is our public key, onto the Web888.

Copy the Key to the Web888

Were are still in our Terminal session and the next task is to set a shell variable with the IP-Address of your Web888's.
ipa=<your Web888's IP-Address>

Note: We will use this shell variable throughout the setup!

Next is a rather long command, which is actually multiple commands strang together which will handle all the required tasks, i.e. setup, copy and to set the appropriate file permissions in one swoop.
cat ~/.ssh/id_rsa.pub | ssh root@$ipa "mkdir -p ~/.ssh && chmod 700 ~/.ssh && cat >> ~/.ssh/authorized_keys && chmod 600 ~/.ssh/authorized_keys"

Note: Make sure the shell variable ipa has been set!

You'll be greeted with this prompt:
root@<your Web888's IP-Address>'s password:
Because it asked so nicely you should give it the password (changeme), hit [ENTER] and the message Connection closed by <your Web888's IP-Address> port 22 will be displayed on the screen. That's it, no more passwords.

Let's check:
ssh root@$ipa "ls  ~/.ssh"
authorized_keys

With that out of the way let's make sure that our modification of the Web888 firmware survives a reboot.

Modify the Web888 Boot Image File

From our Terminal program ssh to the Web888,
ssh root@$ipa
Welcome to Alpine!

The Alpine Wiki contains a large amount of how-to guides and general
information about administrating Alpine systems.
See <https://wiki.alpinelinux.org/>.

You can setup the system with the command: setup-alpine

You may change this message by editing /etc/motd.

Noticed that the Web888 did not ask for a pesky password!

Let's note the Boot Image File size,
ls -l /media/mmcblk0p1/web-888.apkovl.tar.gz
-rwxr-xr-x    1 root     root         13718 Sep 30 00:22 /media/mmcblk0p1/web-888.apkovl.tar.gz

And then check the status of the boot image file,
lbu st
A root/.ssh
A root/.ssh/authorized_keys

As we can see the two files which we created, .ssh and .ssh/authorized_keys have not been added to the boot image file. To add them we commit those files using the command,
lbu ci
And lets check if the commit has worked,
lbu st

And ... nope nada. We get nothing back from the status command if all went well. And to make sure that the boot image file has really been modified we check,
ls -l /media/mmcblk0p1/web-888.apkovl.tar.gz
-rwxr-xr-x    1 root     root         14402 Oct  3 14:53 /media/mmcblk0p1/web-888.apkovl.tar.gz

and see that the timestamp and the files size have changed!

Our job here is done so logout of the Web888,
exit
Connection to <your web888's IP-Address> closed.

It is now time for a cold drink (Beer?). Hard part is done.

Create File Store

Note: Make sure that you are at your local host.

Next we create the file store which is really only a directory at a place near you, you can call it whatever you want, I will call it web888.
mkdir -p ~/web888 

Done!

Test the System

Well, to test the system we will download a message file. I have created a script which will make the download a bit easier. But for demonstration and for testing I show you the individual commands.
  1. Create a copy of the messages file 
  2. Zero/empty /var/log/messages 
  3. Download the log file
  1. ssh root@$ipa 'cp /var/log/messages /var/log/log'
  2. ssh root@$ipa 'cat /dev/null > /var/log/messages'
  3. ssh root@$ipa 'cat /var/log/log' | grep "user\.info" > ~/web888/log_$(date +%Y%m%d"_"%H%M).txt
And we check,
ls ~/web888/log_20251004_0048.txt
/home/hw/web888/log_20251004_0048.txt

and check if our downloaded file has actual data in it,
cat ~/web888/log_20251004_0048.txt | more
Oct  2 13:48:00 web-888 user.info : 00:03:45.107 ..23456789A.     4        WSPR DECODE:  UTC  dB   dT      Freq
 dF  Call   Grid    km  dBm
Oct  2 13:48:00 web-888 user.info : 00:03:45.124 ..23456789A.     4        WSPR DECODE: 1346 -13  0.1 10.140111
 -1  VK7TW  QE37  1057   30 (1.0 W)

Looking good.
The download script can be found here.

And that is it! You should now be able to download the messages (log file) from your Web888 to your local system. The next step is for the tiny bit more advanced users aka the lazy ones like me. This digs into the *nix scheduler which is used for the automation of the download process.

Automate System

Since I'm rather lazy I prefer to automate something as mundane as downloading files. So to automate the download we use a *nix tool called cron. Cron uses a table called a crontab. The following should help you understand crontab. A crontab entry, or a cron expression looks like this,

<minute> <hour> <day-of-month> <month> <day-of-week> <command>

For example the below cron expression is scheduled at 3 minutes past the hour,i.e. it runs at least once every hour and it will run the script /home/hw/web888_transfer.sh.

To list (-l) the crontab use,

crontab -l
3 */1 * * * /home/hw/web888_transfer.sh "192.168.1.40" "/home/hw/web888/log40_$(date +%Y%m%d"-"%H%M).txt" >> /home/hw/tmp/transfer.log 2>&1

and to edit/modify the crontab table we use,

crontab -e

Note: if you are using the crontab -e command for the first time it might ask you what editor you want to use. Now I can't help you with that choice, I do however use vi but ymmv.

Navigate to the bottom of the file [SHIFT-G].
Next type [SHIFT-A], the message 
-- INSERT --
will pop up. You can now type or copy and paste the following,
3 */1 * * * /home/<your username>/web888_transfer.sh "<your web888's IP-Address>" "/home/<your username>/web888/log_$(date +%Y%m%d"-"%H%M).txt" >> /home/<your username>/web888/transfer.log 2>&1

Note: You need to change <your web888's IP-Address> to the actual IP-Address of your Web888, the shell variable does not work here.

Press [CTRL-C] to stop the editing process and then press [SHIFT-ZZ] (not a typo it is double Z)

You'll be greeted with,
crontab: installing new crontab

And that's it. Done and dusted.

Note: You need to download the web888_transfer.sh file into your home directory and make it executable. The following command will make the web888_transfer.sh user executable, 

chmod 700 ~/web888_transfer.sh

Please note that you are free to mangle any of the above to suit your needs. This is something that should work and point you into the right direction. Since I use two Web888 my setup/schedule/script is/are a bit different.

Good luck, and if I've made a(ny) mistake(s) (highly possible) drop me a note and I'll rectify it.

Oh, and a final word! Don't forget that you have to do steps 2 & 3 and maybe the UPDATE below, after you upgraded the OS (firmware) of the Web888. 

UPDATE:

I forgot that the newer *nix's don't allow cron to be used by users so you might need to add your user account to the cron.allow file. To solve this run the below from your login,

sudo sh -c 'who am i | cut -d" " -f 1 > /etc/cron.allow'

And then check,

cat /etc/cron.allow 
<your user name>

And we most likely need to restart the cron daemon. For that we issue this command,

sudo service cron restart

and the check,

sudo service cron status
● cron.service - Regular background program processing daemon
     Loaded: loaded (/lib/systemd/system/cron.service; enabled; vendor preset: enabled)
     Active: active (running) since Sat 2025-10-04 19:30:08 ACST; 7s ago
       Docs: man:cron(8)
   Main PID: 32363 (cron)
      Tasks: 1 (limit: 4915)
        CPU: 15ms
     CGroup: /system.slice/cron.service
             └─32363 /usr/sbin/cron -f


Appendix

A few examples of how to process the log file.

Show all WSPR decodes:
cat web888/sdr40.log | grep "WSPR DECODE" | grep -v " UTC " | awk '{print $1,$2,$3,$12,$13,$14,$15,$16,$17,$18,$19,$20,$21,$22}'

Oct 3 23:20:01 2318 -18 0.3 21.096153 1 VK4GAW QG62NS 1657 23 (200 mW)
Oct 3 23:20:01 2318 -2 0.4 7.040137 0 VK3GOD QF23 583 23 (200 mW)
Oct 3 23:20:01 2318 -26 0.2 21.096146 0 VK4WGR QG62 1629 0 (1000 uW)
Oct 3 23:20:01 2318 -7 0.0 14.097032 0 VK2OP QF69 1452 23 (200 mW)
Oct 3 23:20:02 2318 -13 0.2 14.097101 0 KO4VJV EL89UG 16140 43 (20 W)
Oct 3 23:20:02 2318 -15 0.1 28.126183 -1 WA7X DM49 14017 30 (1.0 W)
Oct 3 23:20:03 2318 -15 0.2 14.097012 0 VK2QQ QF67GT 1338 23 (200 mW)
Oct 3 23:20:03 2318 -18 0.1 28.126169 1 VK6JJJ PH16 2910 23 (200 mW)
Oct 3 23:20:03 2318 -21 0.2 28.126101 0 K5DNR EM13 14984 30 (1.0 W)
Oct 3 23:20:04 2318 -21 2.2 28.126160 0 W3PM EM64 15898 37 (5.0 W)
Oct 3 23:20:04 2318 -22 0.1 28.126087 1 AA7US DM41 13683 23 (200 mW)
Oct 3 23:20:05 2318 -23 0.2 28.126019 1 AD2Q EN81 16371 30 (1.0 W)
Oct 3 23:22:01 2320 11 0.5 18.106102 -2 VK4AAN QG61AH 1475 30 (1.0 W)

Show all FT8 decodes:
cat web888/sdr41.log | grep "FT8 DECODE:" | awk '{print $17" "$18,$19,$21,$20,$12,$13,$14,$15,$16}'

Fri Oct 3 2025 23:33:00 14074.750 KT4DM EM78 -14 16166km
Fri Oct 3 2025 23:33:00 18100.778 WA6VZN FM19 -13 16865km
Fri Oct 3 2025 23:33:00 24916.606 BG8KUB OL39 -15 7956km
Fri Oct 3 2025 23:33:15 24917.353 JA3OOK PM74 -12 7800km
Fri Oct 3 2025 23:33:15 28076.541 JH1BTY PM95 -11 7902km
Fri Oct 3 2025 23:33:30 18101.684 PY3FOX GF49 -14 12601km
Fri Oct 3 2025 23:33:30 18102.344 K5TIA EL29 -11 15004km
Fri Oct 3 2025 23:33:30 28075.403 7L4JWS PM96 -14 8014km
Fri Oct 3 2025 23:33:30 28076.106 RC0JD PO33 -13 9967km
Fri Oct 3 2025 23:33:45 18101.547 JA2YSO/P PM85 -12 7904km
Fri Oct 3 2025 23:33:45 24917.247 W2XX DM26 -13 13569km
Fri Oct 3 2025 23:33:45 24917.494 N0FW EM79 -9 16181km
Fri Oct 3 2025 23:33:45 28076.934 JA8NRS QN12 -11 8692km

Show all FT8 decodes from JA's

cat web888/sdr40.log | grep "FT8 DECODE:" | awk '{print $17" "$18,$19,$21,$20,$12,$13,$14,$15,$16} ' | egrep " J[A-Z][0-9][A-Z][A-Z][A-Z]| J[A-Z][0-9][A-Z][A-Z]"

Fri Oct 3 2025 23:31:15 28076.878 JI1ISA PM95 -13 7902km
Fri Oct 3 2025 23:31:45 21076.228 JK3HFN PM75 -10 7911km
Fri Oct 3 2025 23:31:45 21076.888 JI2VWY PM85 -13 7904km
Fri Oct 3 2025 23:32:00 28076.619 JA0IAA PM97 -9 8125km
Fri Oct 3 2025 23:32:15 18102.459 JF2DEJ PM85 -7 7904km
Fri Oct 3 2025 23:32:45 24915.581 JA2MAX/P PM95 -11 7902km
Fri Oct 3 2025 23:33:15 28076.541 JH1BTY PM95 -11 7902km
Fri Oct 3 2025 23:33:15 24917.353 JA3OOK PM74 -12 7800km
Fri Oct 3 2025 23:33:45 18101.547 JA2YSO/P PM85 -12 7904km
Fri Oct 3 2025 23:33:45 28076.934 JA8NRS QN12 -11 8692km

Show all 40m FT8 decodes
cat web888/sdr40.log | grep "FT8 DECODE:" | awk '{print $17" "$18,$19,$21,$20,$12,$13,$14,$15,$16} ' | grep " 707.\.... "

Fri Oct 3 2025 21:13:00 7076.325 EA2A IN92 -13 16479km
Fri Oct 3 2025 21:13:45 7074.347 EB4SM IN80 -14 16637km
Fri Oct 3 2025 21:14:45 7074.256 B97LRZ/P AE63 +1 4624km
Fri Oct 3 2025 21:15:30 7075.100 IZ7LDC JN80 -12 14967km
Fri Oct 3 2025 21:16:00 7075.866 PE5ZWC MF78 +11 5812km
Fri Oct 3 2025 21:20:00 7075.094 DF2RQ JN57 -13 15551km
Fri Oct 3 2025 21:20:45 7075.634 YC1GCM OI33 -15 4566km
Fri Oct 3 2025 21:43:45 7076.169 EA7KPT IM67 -15 16954km
Fri Oct 3 2025 21:54:45 7076.775 VK6WX OF85 -10 1969km
Fri Oct 3 2025 23:08:30 7076.259 9T5NJE/R HN63 -16 18503km

Tuesday, January 21, 2025

How to log FT8 and WSPR to the system log on the WEB-888

The new Version (v2025.0119) has the ability to write to the system log (Thanks Howard).

NOTE: Depending on the storage size of the used SD-Card, the log file could fill up all usable space. This could impact on system performance and might lead to a stale system.

To enable FT*/WSPR logging to the system go to the Admin interface (http://<your web-888 IP-address>:8073/admin) and select the Extensions Tab.


  • Select either WSPR or FT8 (left grey area),
  • Then set the “Log decodes to syslog” [Yes|No] field to Yes.

As soon as this has been set to yes the system will log to the system log (/var/log/messages).

To check, we can either use the Web-console (Admin interface) or ssh to the web-888 via our favoured ssh-client (I use MobaXterm).

To check if the data is written to the log file.

web-888:~# grep "user.info" /var/log/messages
Jan 21 08:23:31 web-888 user.info : 00:56:31.207 0123456789AB 8 FT8 DECODE: 10137.728 DK9FE JO40 -10 15695km Tue Jan 21 08:23:15 2025
Jan 21 08:23:31 web-888 user.info : 00:56:31.312 0123456789AB 8 FT8 DECODE: 10137.984 IW1CKR JN45 -13 15692km Tue Jan 21 08:23:15 2025
Jan 21 08:23:32 web-888 user.info : 00:56:31.780 0123456789AB 6 FT8 DECODE: 18100.725 JE2GEG PM85 -8 7904km Tue Jan 21 08:23:15 2025
Jan 21 08:23:34 web-888 user.info : 00:56:33.975 0123456789AB 7 FT8 DECODE: 14075.550 BD4VOJ PM01 -11 7681km Tue Jan 21 08:23:15 2025
Jan 21 08:23:46 web-888 user.info : 00:56:46.075 0123456789AB 5 FT8 DECODE: 21075.144 JK4WKO PM54 -6 7831km Tue Jan 21 08:23:30 2025
Jan 21 08:23:46 web-888 user.info : 00:56:46.079 0123456789AB 5 FT8 DECODE: 21075.638 JI1ILB PM96 -10 8014km Tue Jan 21 08:23:30 2025
Jan 21 08:23:59 web-888 user.info : 00:56:59.277 0123456789AB 1 WSPR DECODE: 0822 4 0.2 14.097102 0 VK4TMT QG62 1629 23 (200 mW)
Jan 21 08:23:59 web-888 user.info : 00:56:59.396 0123456789AB 1 WSPR DECODE: 0822 3 0.2 14.097134 0 VK4NE QG62 1629 23 (200 mW)
Jan 21 08:23:59 web-888 user.info : 00:56:59.599 0123456789AB 1 WSPR DECODE: 0822 -14 0.2 14.097073 0 MW0KST IO81 16507 37 (5.0 W)
Jan 21 08:24:00 web-888 user.info : 00:56:59.753 0123456789AB 1 WSPR DECODE: 0822 -15 0.2 14.097018 0 G8MCD IO91 16373 23 (200 mW)

Since we are now writing to the log file constantly it is best to make sure that the log file is being kept at a manageable size. To do so we set up syslogd to roll the log file every so often. For demonstration I’ve setup the syslog configuration file to roll after it reaches 100 kB in size. This size depends on your environment and size of the SD-Card. This is a requirement for your system and as such the roll over size needs to be determined accordingly. 

There are approximately 150 Bytes per line so that should help compute your daily log growth. If the system writes 10 messages per second at an average of 150 Bytes we get about 13 MB a day. 

On my system I have set the log size to 10 MB (10240). This however, can be tailored to individual requirements with up to 99 log files (-b 99)  YMMV!

To set this up follow these steps:

  • ssh to the web-888
  • and change into the config directory
    • cd /etc/conf.d
  • create a backup of the original syslog configuration file

    • cp syslog syslog.orig
  • create a new config file based on log size requirements (-s x) and how many logs we like to keep before overwriting the first (/var/log/messages.0) file (-b z). In the below example I use 100 kB size and rotate up to 7 logs.

    • echo "SYSLOGD_OPTS=\"-s 100 -b 7\"" > syslog
  • check if we got it right

    • cat syslog

SYSLOGD_OPTS="-s 100 -b 7"
  • restart syslog

    • rc-service syslog restart
  • and check if we are logging to the log file

    • grep "user.info" /var/log/messages

And as we can see from the below, the log file rolls over at 100 kB.

web-888:~# ls -l /var/log/messages*
-rw-r-----    1 root     wheel        41316 Jan 21 12:02 /var/log/messages
-rw-r-----    1 root     wheel       102543 Jan 21 11:51 /var/log/messages.0
-rw-r-----    1 root     wheel       102542 Jan 21 11:10 /var/log/messages.1


If you like to see a constant flow of messages use the following command. However, do not use this command in the WEB-Console window. [CTRL]+C doesn’t seem to work and as such the console becomes unusable and you need to restart the web server.

web-888:~# tail -f /var/log/messages
Jan 21 08:23:31 web-888 user.info : 00:56:31.207 0123456789AB 8 FT8 DECODE: 10137.728 DK9FE JO40 -10 15695km Tue Jan 21 08:23:15 2025
Jan 21 08:23:31 web-888 user.info : 00:56:31.312 0123456789AB 8 FT8 DECODE: 10137.984 IW1CKR JN45 -13 15692km Tue Jan 21 08:23:15 2025
Jan 21 08:23:32 web-888 user.info : 00:56:31.780 0123456789AB 6 FT8 DECODE: 18100.725 JE2GEG PM85 -8 7904km Tue Jan 21 08:23:15 2025
Jan 21 08:23:34 web-888 user.info : 00:56:33.975 0123456789AB 7 FT8 DECODE: 14075.550 BD4VOJ PM01 -11 7681km Tue Jan 21 08:23:15 2025
Jan 21 08:23:46 web-888 user.info : 00:56:46.075 0123456789AB 5 FT8 DECODE: 21075.144 JK4WKO PM54 -6 7831km Tue Jan 21 08:23:30 2025
Jan 21 08:23:46 web-888 user.info : 00:56:46.079 0123456789AB 5 FT8 DECODE: 21075.638 JI1ILB PM96 -10 8014km Tue Jan 21 08:23:30 2025
Jan 21 08:23:59 web-888 user.info : 00:56:59.277 0123456789AB 1 WSPR DECODE: 0822 4 0.2 14.097102 0 VK4TMT QG62 1629 23 (200 mW)
Jan 21 08:23:59 web-888 user.info : 00:56:59.396 0123456789AB 1 WSPR DECODE: 0822 3 0.2 14.097134 0 VK4NE QG62 1629 23 (200 mW)
Jan 21 08:23:59 web-888 user.info : 00:56:59.599 0123456789AB 1 WSPR DECODE: 0822 -14 0.2 14.097073 0 MW0KST IO81 16507 37 (5.0 W)
Jan 21 08:24:00 web-888 user.info : 00:56:59.753 0123456789AB 1 WSPR DECODE: 0822 -15 0.2 14.097018 0 G8MCD IO91 16373 23 (200 mW)

The following commands can be used to extract data from the log for further processing:

web-888:~# grep "user.info" /var/log/messages > /dev/shm/user.info

web-888:~# grep "FT8 DECODE" /var/log/messages | sed -e 's/.*FT8 DECODE: \(.*\)2025.*/\1/' > /dev/shm/ft8

web-888:~# grep "FT4 DECODE" /var/log/messages | sed -e 's/.*FT4 DECODE: \(.*\)2025.*/\1/' > /dev/shm/ft4

web-888:~# grep "WSPR DECODE" /var/log/messages | grep -v " WSPR DECODE:  UTC" | sed -e 's/.*WSPR DECODE: \(.*\))*/\1/' > /dev/shm/WSPR

These commands will write all WSPR/FT8/4 messages into shared memory, from which we can now copy the data to other systems for further processing.

NOTE: The system log files WILL be removed when a system reboot is being initiated, that includes a power failure! You will need to make sure to backup your data!

UPDATE: 20250226

To make the changes stick, i.e. survive a reboot etc. we need to commit the changes to the boot config file. See the Alpine Wiki for more info.

The lbu command allows us to make changes to the boot configuration. 

  1. Check the status: 
    • web-888:~# lbu st
    • U etc/conf.d/syslog
    • web-888:~# ls -l /media/mmcblk0p1/web-888.apkovl.tar.gz
    • -rwxr-xr-x    1 root     root         13527 Feb 26 15:04 /media/mmcblk0p1/web-888.apkovl.tar.gz
  2. Commit the change
    • lbu ci
  3. Check to see if the backup worked
    • web-888:~# ls -l /media/mmcblk0p1/web-888.apkovl.tar.gz
    • -rwxr-xr-x    1 root     root         13529 Feb 26 15:08 /media/mmcblk0p1/web-888.apkovl.tar.gz
  4. Reboot the system:
    • web-888:~# sync;reboot 
  5. After the reboot check that your syslog configuration is as expected.
    • web-888:~# cat /etc/conf.d/syslog
    • SYSLOGD_OPTS="-s 100 -b 7"

Saturday, July 13, 2024

Adjusting the S-Meter in HDSDR

Since ICOM has released Firmware v.1.42 for the IC-7610 the I&Q port is working again. This opened up the possibility to use HDSDR (Sampling rate of 1.92MHz with an effective Bandwidth of 1.66MHz) again. Since I still had the S-Meter check setup "set up" from the "Adjusting the S-Meter in Thetis"  I decided to check and adjust, if need be, the HDSDR/IC-7610 combo.

The setup is basically the same as for Thetis, except the SDR in this case is an IC-7610.


Setup:


In HDSDR under Options [F7] we find Calibration Setting. This opens the HDSDR Calibration Panel.




Selecting the S-Meter Calibration tab:



The current configuration seems to correspond to an S-Meter reading of S9 +10dB on HDSDR:

and an S9 on the ICOM without the Pre-Amp engaged.


So next we add -73dBm to the Correct Level [dBm] field and press the [Calculate] button.


And the result is:


reducing attenuation by 6dB we get:


and, as expected, adding 6dB we see:


So in a Software Defined Radio (SDR) application written by Amateur's we do get the proverbial 6dB per S-Unit. 

Friday, July 12, 2024

Adjusting the S-Meter in Thetis

After about five (5) years I resurrected my ANAN 100D again. Trying a few versions, including a development version, I settled on Thetis v2.10.3.5 x64 u2. Seems to be running fine on my Windows 11 system. Quite a few improvements over the last five years. Going through the Setup/Configuration of the system I stumbled over a Level Cal inside the [Calibration] tab which can be found under the [General] tab. This allows one the ability to "automagically" set the S-Meter to a user provided level, .... sweet ....

I've decided to use my trusty old Elecraft XG3 RF Signal Source which I have checked against a calibrated RF Powermeter. At 20m the output at the -33dBm level measured -34.8dBm @ 13.8V. So using an attenuator with 38dB attenuation  will give me a -72.8dBm level into the ANAN. A short RG58 cable into a MFJ-1700B switch and another 50cm of RG58 should compensate for the missing 0.2dB to make it -73dBm.

Setup:
  • Signal generator: XG3
  • Level: -34.8dBm @ 13.8V
  • Att: HP-355C & HP-355D (38dB)




And this is how it look in real live.



Here is the Thetis setup:



After pressing the Level Cal [Start] button, the system goes and runs an internal calibration routine. A window pops up to inform us about the progress status of the calibration.

Well, the result is quite pleasing. 



And if we add 6dB attenuation we get:


And not to forget if we do subtract 6dB attenuation the result is:



Struth, 6dB steps who would have thought that is a possibility. 
Oh and this is at every SSB Bandwidth we choose. My default is 2K1, however if I choose 2K9 the S-Meter still shows -73dBm. Yikes, it is possible! It is software defined after all.

It would be nice if my IC-7610 would not change the S-Meter reading with the engagement of the Pre-Amp(s).



References:

Monday, May 29, 2023

A Multi Frequency WSPRing (whispering) System using KA9Q-Radio

The "KA9Q Radio software suit" is a an ingenious sort of a SDR program suite. It is capable of receiving a huge number of frequencies simultaneously. I believe that this is a unique feature of this applications. It seems to go back to the original *nix philosophy of "small is beautiful" e.g. small well written applications that do one job and do this one job very well. Because there is no graphical ballast, it will perform its task on the most modest comput-hardware. Meaning it will run on a Raspberry Pi system starting with the Pi3. Note, that there is no need for an expensive GPU nor the need to use virtual audio cables or a Soundcard for that matter.

Because of the "multiple frequencies simultaneously" capability I've set up a WSPR receiver for my QTH so that during times that I do not operate I can at least monitor band activity and/or propagation. Initially I will monitor from LF (136kHz) to the end of HF (28MHz). If this is going to workout well I will also start building a system for VHF/UHF.  

Depending on the SDR one is going to use there is a requirement for a fast to really fast USB port. In my case I'm going to use the RX888M2, which needs a really fast USB3 port to stream 32 MHz of bandwidth in one go across the port. However, if I would use my Pluto or RSPdx and any of the TV-Dongles I can get away with a speedy USB2 port and for the SDR-IQ with it max 192kHz a slow USB port will do.

I can't stress this enough, the use of good usb cables is a requirement (not to long and it should have good shielding). Additionally, even though all my cables come with a molded choke, I do add additional choke(s) to all of my cables. I've had good success with that however, if that doesn't help reducing the noise from the USB/PC connection then "these" are apparently very good in suppressing CMC noise (EMI).

Below are the steps that I used to get the system up and running. And I believe that if you follow those steps you will be able to get a working "Multi Frequency WSPR receiving System" up and running in no time.

I've had a spare little PC (NUC) lying around (I7-4765) with 8GB of memory and a 250GB SSD which I bought second hand for about A$250. My installation is based on Debian Version 11.7 (Bullseye) and all following steps are based on this OS. The reason to use the NUC was to have both the SDR and the WSPR decoder running on a single host. The WSPR decoder is chewing up all the available CPU resources. 

If you go with the Pi solution I recommend at least a Pi4-4GB and to install Debian Bullseye.

WARNING: If you are going to use a Raspberry PI as the compute resource make absolutely sure that you have enough current available to support the Pi and the RX888! I use a home made 5V 10A Power-supply which is feed of the shacks 12V rail. I've tried with a powered USB-HUB to no avail!

INFO: I have highlighted text in bold, this is so one can copy and then paste the copied text into a terminal instead of typing everything into the terminal. Don't forget to press <Enter> at the end. Command returns are displayed in courier.

NOTE: sudo didn't work for me out of the box.

There are many ways to fix this issue however, I did it this way:
$ su -
 echo "$(who am i | awk '{print $1}') ALL=(ALL) NOPASSWD: ALL" > /etc/sudoers.d/$(who am i | awk '{print $1}')
chmod 440 /etc/sudoers.d/$(who am i | awk '{print $1}')
# ls /etc/sudoers.d/$(who am i | awk '{print $1}')
 -r--r----- 1 root root 27 May 28 20:21 /etc/sudoers.d/<your username>

So with that out of the way we can now get and install required libraries.

$ echo "
build-essential
libusb-1.0-0-dev
libusb-dev
libncurses5-dev
libfftw3-dev
libbsd-dev
libhackrf-dev
libopus-dev
libairspy-dev
libairspyhf-dev
librtlsdr-dev
libiniparser-dev
libavahi-client-dev
portaudio19-dev
libopus-dev" > /tmp/required

$ for _required in $(cat /tmp/required); do sudo apt install $_required -y; done

If you like to run the system for a while it helps if the stupid hibernation is switched off.

 $ sudo systemctl mask sleep.target suspend.target hibernate.target hybrid-sleep.target

I find it much easier to work with names rather then with ip-addresses and port-numbers, as such I'd use the Multicast DNS function of the AVAHI Daemon. Your mileage may vary though! 

$ sudo systemctl start avahi-daemon.service
$ sudo systemctl status avahi-daemon.service
 avahi-daemon.service - Avahi mDNS/DNS-SD Stack
     Loaded: loaded (/lib/systemd/system/avahi-daemon.service; enabled; vendor preset: enabled)
     Active: active (running) since Sun 2023-06-11 22:16:53 ACST; 3 days ago
TriggeredBy: ● avahi-daemon.socket
   Main PID: 548 (avahi-daemon)
     Status: "avahi-daemon 0.8 starting up."
      Tasks: 2 (limit: 9342)
     Memory: 1.3M
        CPU: 15.899s
     CGroup: /system.slice/avahi-daemon.service
             ├─548 avahi-daemon: running [sun.local]
             └─566 avahi-daemon: chroot helper

Not needed in the use case.

Check if en_US.UTF-8 language is installed.
First check:
$ locale -a
C
C.UTF-8
en_AU.utf8
POSIX

And if it is missing, run:
$ sudo dpkg-reconfigure locales

This will bring up a graphical-ui (ncurses) selection display. Scroll down until you see en_US_UTF-8, press spacebar to select and then <ok>

Generating locales (this might take a while...)
  en_AU.UTF-8... done
  en_US.UTF-8... done
Generation complete. 
 
$ locale -a
C
C.UTF-8
en_AU.utf8
en_US.utf8 <-- required
POSIX

Now download the source files for the KA9Q radio suit:
$ mkdir tmp
$ cd tmp
wget https://github.com/ka9q/ka9q-radio/archive/refs/heads/main.zip

Extract the downloaded file (main.zip):
$ unzip main.zip
$ ls -l
total 2.4M
drwxr-xr-x 4 hw hw 4.0K May 28 16:15 ka9q-radio-main
-rw-r--r-- 1 hw hw 2.4M May 28 18:16 main.zip

Compile and install KA9Q-Radio applications:
$ export LANG=en_US.UTF-8
$ cd ka9q-radio-main

NOTE: There are two (2) Makefiles, Makefile.linux and Makefile.pi. If you use a RaPi you need to link Makefile.pi to Makefile 

$ ln -s Makefile.linux Makefile 
$ sudo make install

Next we create a "wisdom" file. 

NOTE: Depending on your system this can take a while. Creating a wisdom file on a RaPi took a while. I cooked a meal, ate the meal, check the RaPi .... not yet .... went and watch a movie ... check again and voila done! So be patience grasshopper.

$ fftwf-wisdom -v -T 1 -o /tmp/wisdom rof500000 cof36480 cob1920 cob1200 cob960 cob800 cob600 cob480 cob320 cob300 cob200 cob160

Or if you are keen and like to know how long it took to create your wisdom file you could run the below.

$ time fftwf-wisdom -v -T 1 -o /tmp/wisdom rof500000 cof36480 cob1920 cob1200 cob960 cob800 cob600 cob480 cob320 cob300 cob200 cob160
fftw-wisdom: system-wisdom import failed
Planning transform: cob160
Planning transform: cob200
Planning transform: cob300
Planning transform: cob320
Planning transform: cob480
Planning transform: cob600
Planning transform: cob800
Planning transform: cob960
Planning transform: cob1200
Planning transform: cob1920
Planning transform: cof36480
Planning transform: rof500000

real    4m14.131s
user    4m8.088s
sys     0m0.048s

Move the generated wisdom file in to its rightful place.
$ test -s /var/lib/ka9q-radio/wisdom && sudo mv /var/lib/ka9q-radio/wisdom /var/lib/ka9q-radio/wisdom.old
$ sudo mv /tmp/wisdom /var/lib/ka9q-radio/wisdom
$ sudo chown $(who am i | awk '{print $1}').radio /var/lib/ka9q-radio/wisdom

Now its time to install the K1JT's WSPR Application.
$ sudo apt install wsjtx -y

After that is finished it's time ....  to configure/setup KA9Q-Radio for our intended purpose.

There are lots of config files and to work through all off them you'll want/need to read the documentation from here however, since I'm going to use the RX888 to do WSPR only as such I've created a new rx888.conf config file. The # and ; a comment markers. You can either modify this file with a text editor like vi or nano.

Before you copy and paste the next configuration item (below, you need to establish what the active network interface is being named! In the rx888.conf file we has a statement starting with iface. This value is system dependent and you need to find that name before you can progress any further.

On a RaPi it is most likely eth0, but on other systems it could be something else. E.g. on my system it is eno1
You'll be able to check using the following:
ip a | grep "mtu 1500" | grep UP | awk '{print $2}'
eno1: <- This is what the NIC is called on my system

Whatever the above command output shows, you'll might need to change the iface value in the below script. E.g. if it shows eth0 your iface line should read iface = eth0.

The below will create a backup of the /etc/radio/rx888d.conf file first, before creating a new /etc/radio/rx888d.conf file. 

$ sudo mv /etc/radio/rx888d.conf /etc/radio/rx888d.conf.bck
$ sudo cat << __EOF__ > /etc/radio/rx888d.conf
[rx888-loop]
# VK5HW customized
description = "RX888 40m Delta-Loop"
firmware = SDDC_FX3.img
samprate = 64800000    ;  2^8 * 3^4 * 5^5
iface = eth0               ; replace this with your iface name
status = rx888-status.local
data = rx888-pcm.local
ssrc = 10
;gain = 1.5 ; dB
gain = 10 ;dB - close to the Noise Floor, might have to increase
gainmode = high ; higher gain range
__EOF__

Next we need to configure the virtual receivers. These we do with the radiod@wspr.conf file. 
$ sudo mv /etc/radio/radiod@wspr.conf /etc/radio/radiod@wspr.conf.old
$ sudo cat << __EOF__ > /etc/radio/radiod@wspr.conf
[global]
overlap = 5
blocktime = 20
input = rx888-status.local
samprate = 12000
mode = usb
status = hf.local
fft-threads = 2

[WSPR]
# Bottom of 200 Hz WSPR segments on each band. Center is 1500 Hz higher
# sample rate must be 12 kHz as required by wsprd
data = wspr-pcm.local
freq = "136k000 474k200 1m836600 3m568600 5m287200 7m038600 10m138700 14m095600 18m104600 21m094600 24m924600 28m124600"
__EOF__

That's it folks, we are ready to start KA9Q-Radio! Make sure the RX888 is plug into the correct USB Port and then to be sure to be sure REBOOT the system.
$ sudo reboot

After the reboot, login to the system and start the rx888d(river) using the rx888-loop configuration.
/usr/local/sbin/rx888d rx888-loop &

NOTE: The & indicates that we would like the program (job) to run in the background.

If everything is OK the following output can be seen:
$ Using config file /etc/radio/rx888d.conf
Loading firmware file /usr/local/share/ka9q-radio//SDDC_FX3.img
Firmware already loaded
USB speed: 4
Successfully claimed interface
Samprate 64,800,000, Gain 10.0 dB, Attenuation 0.0 dB, Dithering 0, Randomizer 0, USB Queue depth 16, USB Request size 8 * pktsize 16384 = 131,072 bytes
service 'RX888 40m Delta-Loop._ka9q-ctl._udp' -> rx888-status.local (239.132.105.12) established
service 'RX888 40m Delta-Loop._rtp._udp' -> rx888-pcm.local (239.10.102.92) established
RX888 40m Delta-Loop: iface eno1; status -> 239.132.105.12:5006, data -> 239.10.102.92:5004 (TTL 0, TOS 48, 24576 samples/packet)

If this all looks ok the PC is talking to the SDR and it is time to start the demodulators.
$ /usr/local/sbin/radiod /etc/radio/radiod@wspr.conf &
$ KA9Q Multichannel SDR
Copyright 2018-2022 by Phil Karn, KA9Q; may be used under the terms of the GNU General Public License
Loading config file /etc/radio/radiod@wspr.conf...
Acquired front end control stream rx888-status.local (239.132.105.12)
Acquired front end data stream 239.10.102.92:5004 (239.10.102.92)
Front end sample rate 64,800,000 Hz, real; block time 20.0 ms, 50.0 Hz
fftwf_import_system_wisdom() failed
fftwf_import_wisdom_from_filename(/var/lib/ka9q-radio/wisdom) succeeded
service 'wspr._ka9q-ctl._udp' -> hf.local (239.83.95.156) established
Processing [wspr]
service 'wspr._rtp._udp' -> wspr-pcm.local (239.72.24.12) established
12 demodulators started
12 total demodulators started

Next start start the wspr-decoder:
$ wspr-decoded wspr-pcm.local

And voila we are starting to see decodes:
$ <DecodeFinished>
<DecodeFinished>
<DecodeFinished>
<DecodeFinished>
<DecodeFinished>
<DecodeFinished>
1518 -19  0.7   0.475683  0  VK6LX OF88 30
<DecodeFinished>
<DecodeFinished>
<DecodeFinished>
1518 -17  0.3   7.040113  0  RU0LL PN53 33
<DecodeFinished>
1518 -27  0.4  14.096975  0  WS5L EM13 37
1518 -12  0.2  14.096984  0  WB7AJP CN87 33
1518 -23  0.3  14.097017  0  <KR6RG> DM13EM 23
<DecodeFinished>
<DecodeFinished>

Looking at our home directory we can see that we have a few new directories. These are the directories of the individual virtual receivers where all the WSPR files go.
$ ls -l
total 104K
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 10138700
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 136000
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 14095600
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 18104600
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 1836600
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 21094600
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 24924600
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 28124600
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 3568600
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 474200
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 5287200
drwxr-xr-x 2 hw hw 4.0K May 29 00:54 7038600

Lets have a look what's inside these directories:
$ ls -l 5287200
total 8.7M
-rwxr-xr-x 1 hw hw 2.1M May 28 00:01 230527_1430.wav
-rwxr-xr-x 1 hw hw 1.9M May 28 01:17 230527_1546.wav
-rwxr-xr-x 1 hw hw 1.4M May 28 09:05 230527_2334.wav
-rwxr-xr-x 1 hw hw 2.6M May 29 00:47 230528_1516.wav
-rwxr-xr-x 1 hw hw 2.1M May 29 00:57 230528_1526.wav
-rw-r--r-- 1 hw hw    0 May 28 00:33 ALL_WSPR.TXT
-rw-r--r-- 1 hw hw    0 May 29 00:55 hashtable.txt
-rw-r--r-- 1 hw hw    0 May 29 00:55 wspr_spots.txt
-rw-r--r-- 1 hw hw  452 May 29 00:55 wspr_timer.out
-rw-r--r-- 1 hw hw 2.0K May 29 00:55 wspr_wisdom.dat
$ ls -l 14095600
total 8.8M
-rwxr-xr-x 1 hw hw 2.1M May 28 00:01 230527_1430.wav
-rwxr-xr-x 1 hw hw 1.9M May 28 01:17 230527_1546.wav
-rwxr-xr-x 1 hw hw 1.4M May 28 09:05 230527_2334.wav
-rwxr-xr-x 1 hw hw 2.6M May 29 00:47 230528_1516.wav
-rwxr-xr-x 1 hw hw 2.1M May 29 00:59 230528_1528.wav
-rw-r--r-- 1 hw hw  36K May 28 23:34 all_wspr.bck
-rw-r--r-- 1 hw hw 3.0K May 29 00:57 ALL_WSPR.TXT
-rw-r--r-- 1 hw hw 3.6K May 29 00:57 hashtable.txt
-rw-r--r-- 1 hw hw  45K May 28 23:34 upload.log
-rw-r--r-- 1 hw hw   74 May 29 00:57 wspr_spots.txt
-rw-r--r-- 1 hw hw  452 May 29 00:57 wspr_timer.out
-rw-r--r-- 1 hw hw 2.0K May 29 00:57 wspr_wisdom.dat

In the 14095600 directory we find two additional files, all_wspr.bck and upload.log. These are files that my upload script is producing. The .bck file is an archive of ALL_WSPR.TXT and the .log file is for error checking.

And here is a snapshot view of a decoded dataset:

$ cat 14095600/ALL_WSPR.TXT
230528 1508 -18 -0.91  14.0969794  DP0POL JQ26 37          0  0.12  3  1    0  0  34    33  -120
230528 1508 -15  0.19  14.0969838  WB7AJP CN87 33          1  0.28  1  1    0  0  39    20    16

As it stands you now have a MULTI BAND WSPR DECODING SYSTEM

And now to upload the results to WSPRNet, that is also quite easy. It is  done with a one-liner:

$ curl -F allmept=@ALL_WSPR.TXT  -F call=<your call>  -F grid=<your grid>  http://wsprnet.org/meptspots.php

There is a very similar site, PSKREPORTER which not only has a database for WSPR but for a lot of other digital modes. However, I've not (yet) found a way how to upload my results to that repository.

I have automated this on my system using the *nix cron facility. I first created a script to archive the reports and create a log file for error checking. If you like to use the script, it can be found here

And here is my crontab: (copy the below to your crontab)
1,7,13,19,25,31,37,43,49,55 * * * * ~/scripts/upload_wspr_data.sh VK5HW PF94hk

It basically runs every 6 minutes, a minute after or before a WSPR decode.

Oh, and if you like to see if I received you go to WSPR rocks, which is an excellent tool for mapping, charting and other visualisation. 

So that's basically it. 
PLEASE NOTE, that this is NOT a how to use KA9Q-Radio, rather a quick way to show how easy it is to setup a multiple frequency WSPR receiving system using parts of the KA9Q-Radio suite. 
There are other modes that can be setup and monitored,  if you like to use other modes or even would like to manually tune through the bands and listen you'd need either a sound card or understand how to use "MULTICAST" streaming. The options are not quite endless but ...

Next on the list, get some horizontal antenna for a 2m/70cm/23cm installed and setup a WSPR VHF/UHF receive system. 

Oh and a little bragging ...





Appendix: