Friday 23 September 2022

Fight Internet Censorship - Run a Snowflake Proxy

 Why run a snowflake proxy?

News reports indicate that current events caused a sharp increase in blocked or limited internet accessibility in several countries. One option to circumvent these measures is to use a TOR browser. Also available on mobile phones - e.g. Onion Browser for iOS.

In the cable tray under my desk, there lives a Raspberry Pi

As one method to aid circumventing DNS, IP and routing measures, you can run a snowflake proxy. Your PC or server temporarily registers with a public broker as a snowflake proxy. A user's TOR browser will ask the broker for the address of a snowflake proxy, which will give access to the TOR network. Snowflake is one of several bridging techniques that can be chosen in the browser if it can't connect to the TOR network directly.

Snowflake Schematic (originally from here)

Can I get in trouble for running a snowflake proxy?

If you live in a healthy democracy, it is most unlikely to get into trouble for running a snowflake proxy. Other than running a TOR Exit Node (very bad idea!), your IP address won't show up in any web server logs.

Can anyone do bad things over my connection?

Yes. But you will never know, as all communication is encrypted. Like any technology, TOR can be used for good and bad. - I decided that at this moment the benefits outweigh the risks. I hope that will change some time soon.

If you don't run a dedicated server, consider running snowflake as a plugin for Firefox or chrome, so you can help while surfing the web.

Will it eat up my Internet bandwidth?

That depends. It does not even remotely for me, but I have the luxury of a super stable 100/40 MBit/s DSL connection.

Set up Snowflake on a Raspberry Pi

There are several tutorials on the web, but none worked flawlessly 1st try. So here is my take. I assume you have a basic knowledge of Linux/Unix.

Fetch the current version of the GO programming language for the Raspi:

me@myhost:$ wget https://go.dev/dl/go1.19.1.linux-armv6l.tar.gz

--2022-09-22 14:32:59--  https://go.dev/dl/go1.19.1.linux-armv6l.tar.gz
.... much Output deleted .....
go1.19.1.linux-armv6l.tar 100%[===================================>] 111,29M  1,38MB/s    in 33s
2022-09-22 14:33:32 (3,42 MB/s) - »go1.19.1.linux-armv6l.tar.gz« saved [116690970/116690970]

Unpack that to /usr/local (or wherever you like it best):

me@myhost:$ sudo tar -C /usr/local -xzf go1.19.1.linux-armv6l.tar.gz

You can get rid og the archive to free up the space:

me@myhost:$ rm go1.19.1.linux-armv6l.tar.gz

Add the location of the go binary to your path to make things easy:

me@myhost:$ vi ~/.profile

At the end of the file simply add these two lines:

PATH=$PATH:/usr/local/go/bin
GOPATH=$HOME/go

Save that (ESC : w q)

You can now either log off and on again, or type

me@myhost:$ source ~/.profile

to update the environment variables.

If you haven't installed git already, do so by typing:

me@myhost:$ sudo apt install git

If you have a preferred directory where you build your stuff, cd to there. Otherwise just type:

me@myhost:$ git clone https://git.torproject.org/pluggable-transports/snowflake.git

Then change to the proxy subdirectory:

me@myhost:$ cd snowflake
me@myhost:$ cd proxy

And build the snowflake proyx:

me@myhost:$ go build

Once that has completed, you can run it right from where you are:

me@myhost:~/snowflake/proxy $ ./proxy -verbose

2022/09/24 14:01:55 starting
2022/09/24 14:01:56 WebRTC: Created offer
2022/09/24 14:01:56 WebRTC: Set local description
2022/09/24 14:01:56 Offer:... OUTPUT REMOVED
2022/09/24 14:02:25 NAT Type measurement: unknown -> restricted = restricted
2022/09/24 14:02:25 NAT type: restricted
2022/09/24 14:02:35 sdp offer successfully received.
2022/09/24 14:04:48 OnDataChannel
2022/09/24 14:04:48 Connection successful.
2022/09/24 14:04:48 OnOpen channel
2022/09/24 14:10:56 connected to relay: wss://snowflake.torproject.net/
2022/09/24 14:10:58 OnClose channel
2022/09/24 14:10:58 Traffic throughput (up|down): 8 KB|11 KB -- (43 OnMessages, 16 Sends, over 15 seconds)
2022/09/24 14:10:58 copy loop ended
2022/09/24 14:10:58 datachannelHandler ends

It might take a while for someone to actually connect. Don't worry if you see time-outs.

CTRL-C ends the process. On a Raspberry Pi, you'll probably want to run the proxy even while logged off. My way of doing that is not what you'd want for a permanent installation. As I do a lot of experimental stuff on the Raspi, I like to manually start what I want to run:

me@myhost:$ nohup ./proxy > ~/snowflake.log 2>&1 &

That has the downside that you manually have to kill the process when you want to stop it. 

me@myhost:$ kill -9 $(pidof proxy)

Some extra load on the Raspi

The proxy takes a fair bit of CPU power from the Raspberry. But it never even comes close to using the full capacity and the sub processes balance nicely over the four cores of my somewhat aged pi.

Moderate extra load on the Raspberry Pi

Sniffing snowflake traffic on your network

You can actually find out where the users using your proxy are located. As the clients connect (well, it is UDP, so strictly speaking "connect" is not quite right) to your proxy, you can use tcpdump for a quick analysis:

me@myhost:$ sudo tcpdump -i eth0 -s 1500 port not 22 and port not 53

On the output, look out for UDP traffic. There will be some lines looking like this:
18:09:46.914455 IP myhost.mydomain.de.60101 > 188.208.20Y.XXX.39043: UDP, length 1265
18:09:46.914894 IP myhost.mydomain.de.60101 > 188.208.20Y.XXX.39043: UDP, length 269
18:09:46.915551 IP myhost.mydomain.de.60101 > 188.208.20Y.XXX.39043: UDP, length 1265
18:09:46.915987 IP myhost.mydomain.de.60101 > 188.208.20Y.XXX.39043: UDP, length 269
18:09:46.916724 IP myhost.mydomain.de.60101 > 188.208.20Y.XXX.39043: UDP, length 1265
18:09:46.917321 IP myhost.mydomain.de.60101 > 188.208.20Y.XXX.39043: UDP, length 269
18:09:46.918309 IP myhost.mydomain.de.60101 > 188.208.20Y.XXX.39043: UDP, length 1265
18:09:46.919010 IP myhost.mydomain.de.60101 > 188.208.20Y.XXX.39043: UDP, length 269
18:09:46.919814 IP myhost.mydomain.de.60101 > 188.208.20Y.XXX.39043: UDP, length 1265

To protect the user, I edited out the full IP.
The IP (or possibly hostname, if it can be resolved) can be fed to the excellent Domain Dossier web site from Centralops.net for analysis.

The Network whois record will show who owns that network and where the company is located. In this case it is (no wonder at this moment) in Teheran, Iran.
That result was to be expected
From a brief observation, I see almost exclusively IPs from providers in Iran, Iraq and Russia.

The slowflake log

When not in verbose mode, the proxy logs to stdout (or in my case to snowflake.log) hourly and reports some statistics:
me@myhost:~ $ tail -f snowflake.log
2022/09/18 04:12:49 In the last 1h0m0s, there were 7 connections. Traffic Relayed ↑ 1 GB, ↓ 87 MB.
2022/09/18 05:12:49 In the last 1h0m0s, there were 8 connections. Traffic Relayed ↑ 3 MB, ↓ 487 KB.
2022/09/18 06:12:49 In the last 1h0m0s, there were 20 connections. Traffic Relayed ↑ 327 MB, ↓ 34 MB.
2022/09/18 07:12:49 In the last 1h0m0s, there were 17 connections. Traffic Relayed ↑ 14 MB, ↓ 4 MB.
2022/09/18 08:12:49 In the last 1h0m0s, there were 34 connections. Traffic Relayed ↑ 46 MB, ↓ 18 MB.
2022/09/18 09:12:49 In the last 1h0m0s, there were 15 connections. Traffic Relayed ↑ 23 MB, ↓ 12 MB.
2022/09/18 10:12:49 In the last 1h0m0s, there were 40 connections. Traffic Relayed ↑ 286 MB, ↓ 70 MB.
2022/09/18 11:12:49 In the last 1h0m0s, there were 19 connections. Traffic Relayed ↑ 46 MB, ↓ 15 MB.
2022/09/18 12:12:49 In the last 1h0m0s, there were 27 connections. Traffic Relayed ↑ 134 MB, ↓ 35 MB.
2022/09/18 13:12:49 In the last 1h0m0s, there were 13 connections. Traffic Relayed ↑ 32 MB, ↓ 3 MB.

The traffic volume varies widely from a few MBytes s to a GByte per hour.


Friday 24 June 2022

Fixing broken VNC dependencies on Raspberry PI

 Out of the blue, when updating my Raspberry threw this error:

users@host:~ $ sudo apt-get upgrade

Paketlisten werden gelesen... Fertig

Abhängigkeitsbaum wird aufgebaut.

Statusinformationen werden eingelesen.... Fertig

Paketaktualisierung (Upgrade) wird berechnet... Fertig

Einige Pakete konnten nicht installiert werden. Das kann bedeuten, dass

Sie eine unmögliche Situation angefordert haben oder, wenn Sie die

Unstable-Distribution verwenden, dass einige erforderliche Pakete noch

nicht erstellt wurden oder Incoming noch nicht verlassen haben.

Die folgenden Informationen helfen Ihnen vielleicht, die Situation zu lösen:


Die folgenden Pakete haben unerfüllte Abhängigkeiten:

 vlc-bin : Hängt ab von: libvlc-bin (= 3.0.12-0+deb10u1+rpt3) aber 3.0.17.4-0+deb10u1 soll installiert werden

 vlc-plugin-base : Hängt ab von: vlc-data (= 3.0.12-0+deb10u1+rpt3) aber 3.0.17.4-0+deb10u1 soll installiert werden

 vlc-plugin-skins2 : Hängt ab von: vlc-plugin-qt (= 3.0.17.4-0+deb10u1) aber 3.0.12-0+deb10u1+rpt3 soll installiert werden

E: Beschädigte Pakete

Damaged packets? Broken depencencies? Her is what helped:

user@host:~ $ sudo apt-get install --reinstall vlc-bin

And after a little while sudo apt-get upgrade ran without any issues

Tuesday 11 January 2022

Arduino HTTP client connection through corporate proxy server

Connecting an Arduino to the web via a http proxy

http Proxies

There are a variety of reasons why you'd want to have a http-Proxy in a corporate environment. Security, content filtering, authentication and authorization, as well as logging and traffic optimization.
And it makes a lot of sense to not point the default gateway of a corporate network towards the internet, as this raises the bar for malicious software (and users) to bypass network security. But that means that software that needs to access the internet has to do so through the proxy server.
This example has been tried through a SQUID proxy with more or less default configuration.

Arduino http client

The Arduino ethernet library provides sufficient functionality to get basic tasks done. In this case get the weather feed from BBC for a specific location. But with a little twist: It only speaks to the proxy server.
Example Setup

Modified "Web Client" example

This is just a slightly modified version of the "Examples->Ethernet->WebClient" that comes with the Arduino IDE and an installed Ethernet library. Works as show with an Arduino MKR Zero, but should work with an UNO just as well.


 /*  
  Web proxy client  
  This sketch reads an RSS feed from a website  
  using an Arduino Wiznet Ethernet shield.  
  Circuit:  
  * Ethernet shield attached to pins 10, 11, 12, 13  
  created 18 Dec 2009  
  by David A. Mellis  
  modified 9 Apr 2012  
  by Tom Igoe, based on work by Adrian McEwen  
  modified 11 Jan 2022  
  by Andy Reischle for http proxy support  
  */  
 #include <SPI.h>  
 #include <Ethernet.h>  
 // Enter a MAC address for your controller below.  
 // Newer Ethernet shields have a MAC address printed on a sticker on the shield  
 // Don't forget to change that when you have more than one of these on one network segment  
 byte mac[] = { 0xDE, 0xAD, 0xBE, 0xEF, 0xFE, 0xED };  
 // if you don't want to use DNS (and reduce your sketch size)  
 // use the numeric IP instead of the name for the server:  
 //IPAddress server(74,125,232,128); // numeric IP for Google (no DNS)  
 char server[] = "weather-broker-cdn.api.bbci.co.uk";  // name address for target server (using DNS)  
 char proxy[] = "proxy.internal.mycorporation.something"; // name of the internal proxy server (using DNS)  
 // Set the static IP address to use if the DHCP fails to assign  
 // That will be rather pointless in a corporate environment  
 IPAddress ip(192, 168, 0, 177);  
 IPAddress myDns(192, 168, 0, 1);  
 // Initialize the Ethernet client library  
 // with the IP address and port of the server  
 // that you want to connect to (port 80 is default for HTTP):  
 EthernetClient client;  
 // Variables to measure the speed  
 unsigned long beginMicros, endMicros;  
 unsigned long byteCount = 0;  
 bool printWebData = true; // set to false for better speed measurement  
 void setup() {  
  // You can use Ethernet.init(pin) to configure the CS pin  
  //Ethernet.init(10); // Most Arduino shields  
  Ethernet.init(5);  // MKR ETH shield  
  //Ethernet.init(0);  // Teensy 2.0  
  //Ethernet.init(20); // Teensy++ 2.0  
  //Ethernet.init(15); // ESP8266 with Adafruit Featherwing Ethernet  
  //Ethernet.init(33); // ESP32 with Adafruit Featherwing Ethernet  
  // Open serial communications and wait for port to open:  
  Serial.begin(9600);  
  while (!Serial) {  
   ; // wait for serial port to connect. Needed for native USB port only  
  }  
  // start the Ethernet connection:  
  Serial.println("Initialize Ethernet with DHCP:");  
  if (Ethernet.begin(mac) == 0) {  
   Serial.println("Failed to configure Ethernet using DHCP");  
   // Check for Ethernet hardware present  
   if (Ethernet.hardwareStatus() == EthernetNoHardware) {  
    Serial.println("Ethernet shield was not found. Sorry, can't run without hardware. :(");  
    while (true) {  
     delay(1); // do nothing, no point running without Ethernet hardware  
    }  
   }  
   if (Ethernet.linkStatus() == LinkOFF) {  
    Serial.println("Ethernet cable is not connected.");  
   }  
   // try to configure using IP address instead of DHCP:  
   Ethernet.begin(mac, ip, myDns);  
  } else {  
   Serial.print(" DHCP assigned IP ");  
   Serial.println(Ethernet.localIP());  
  }  
  // give the Ethernet shield a second to initialize:  
  delay(1000);  
  Serial.print("connecting to ");  
  Serial.print(proxy);  
  Serial.println("...");  
  // if you get a connection, report back via serial:  
  // target the request to the proxy  
  if (client.connect(proxy, 8080)) {  
   Serial.print("connected to ");  
   Serial.println(client.remoteIP());  
   // Make a HTTP request to the proxy:  
   client.println("GET http://weather-broker-cdn.api.bbci.co.uk/en/observation/rss/2907669 HTTP/1.1");  
   client.println("Host: weather-broker-cdn.api.bbci.co.uk");  
   client.println("Connection: close");  
   client.println();  
  } else {  
   // if you didn't get a connection to the server:  
   Serial.println("connection failed");  
  }  
  beginMicros = micros();  
 }  
 void loop() {  
  // if there are incoming bytes available  
  // from the server, read them and print them:  
  int len = client.available();  
  if (len > 0) {  
   byte buffer[80];  
   if (len > 80) len = 80;  
   client.read(buffer, len);  
   if (printWebData) {  
    Serial.write(buffer, len); // show in the serial monitor (slows some boards)  
   }  
   byteCount = byteCount + len;  
  }  
  // if the server's disconnected, stop the client:  
  if (!client.connected()) {  
   endMicros = micros();  
   Serial.println();  
   Serial.println("disconnecting.");  
   client.stop();  
   Serial.print("Received ");  
   Serial.print(byteCount);  
   Serial.print(" bytes in ");  
   float seconds = (float)(endMicros - beginMicros) / 1000000.0;  
   Serial.print(seconds, 4);  
   float rate = (float)byteCount / seconds / 1000.0;  
   Serial.print(", rate = ");  
   Serial.print(rate);  
   Serial.print(" kbytes/second");  
   Serial.println();  
   // do nothing forevermore:  
   while (true) {  
    delay(1);  
   }  
  }  
 }  
If all goes well, this is the output:

Initialize Ethernet with DHCP:
  DHCP assigned IP 1.2.3.4
connecting to proxy.internal.mycorporation.something...
connected to 4.3.2.1
HTTP/1.0 200 OK
Access-Control-Allow-Origin: *
Content-Type: application/rss+xml
ETag: "8b0e69972930952dc99c4d239b1c489402f5940392a0bbe16ed534b0b242787f"
expiry_extended_seconds: 0
Server: Apache/2.4.6 (CentOS) OpenSSL/1.0.2k-fips
Cache-Control: no-transform, max-age=60
Date: Tue, 11 Jan 2022 18:37:23 GMT
Content-Length: 1660
X-Cache: MISS from proxy.internal.mycorporation.something
X-Cache-Lookup: MISS from proxy.internal.mycorporation.something:8080
Via: 1.0 proxy.internal.mycorporation.something:8080 (squid/2.6.STABLE21)
Proxy-Connection: close

<?xml version="1.0" encoding="UTF-8"?>
<rss xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/" xmlns:georss="http://www.georss.org/georss" version="2.0">
  <channel>
    <title>BBC Weather - Observations for  Heilbronn, DE</title>
    <link>https://www.bbc.co.uk/weather/2907669</link>
    <description>Latest observations for Heilbronn from BBC Weather, including weather, temperature and wind information</description>
    <language>en</language>
    <copyright>Copyright: (C) British Broadcasting Corporation, see http://www.bbc.co.uk/terms/additional_rss.shtml for more details</copyright>
    <pubDate>Tue, 11 Jan 2022 17:00:00 GMT</pubDate>
    <dc:date>2022-01-11T18:00:00Z</dc:date>
    <dc:language>en</dc:language>
    <dc:rights>Copyright: (C) British Broadcasting Corporation, see http://www.bbc.co.uk/terms/additional_rss.shtml for more details</dc:rights>
    <atom:link href="https://weather-service-thunder-broker.api.bbci.co.uk/en/observation/rss/2907669" type="application/rss+xml" rel="self" />
    <item>
      <title>Tuesday - 18:00 CET: Not available, 4°C (40°F)</title>
      <link>https://www.bbc.co.uk/weather/2907669</link>
      <description>Temperature: 4°C (40°F), Wind Direction: Easterly, Wind Speed: 4mph, Humidity: 66%, Pressure: 1033mb, Rising, Visibility: --</description>
      <pubDate>Tue, 11 Jan 2022 18:00:00 GMT</pubDate>
      <guid isPermaLink="false">https://www.bbc.co.uk/weather/2907669-2022-01-11T18:00:00.000+01:00</guid>
      <dc:date>2022-01-11T17:00:00Z</dc:date>
      <georss:point>49.1399 9.2205</georss:point>
    </item>
  </channel>
</rss>

disconnecting.
Received 2181 bytes in 0.4765, rate = 4.58 kbytes/second

No big deal extracting values from that output. I've done something similar here:
Fritzmeter Project , but with TinyXML you should be able to do that with more style.

The proxy log (acces.log) entry shows:
1.2.3.4 TCP_MISS/200 2181 GET http://weather-broker-cdn.api.bbci.co.uk/en/observation/rss/2907669 - DIRECT/184.86.251.134 application/rss+xml


What about HTTPS?

The Arduino Uno is to anemic for HTTPS, but the experiments on this page have been done with a MKR ZERO and the matching MKR ETH Shield. The MKR ZERO can do HTTPS easily.
But:
HTTPS through a http-proxy works differently from HTTP: The client requests a tunneled connection from the Proxy, using the "CONNECT" method (rather than "GET" in the HTTP example).
The otherwise brilliant SSLClient library does not provide an obvious way to support that:
I have not yet found a way to establish that first part of the connection request without encryption and switch to SSL/TLS after the tunnel is initiated.