Wednesday, 23 September 2015

Generic UDP proxy for NodeMCU / ESP8266 - Simple LUA DNS proxy

For a project that has been in the works for quite some time, I need a ESP8266 module to act as a DNS proxy. Other than on my very popular CaptiveIntraweb project, that simply lies to all DNS requests, I need real DNS lookups this time.

Getting my head slowly around the event driven nature of NodeMCU, the code turned into a very compact, generic UDP proxy or forwarder.
The script is completely unaware of the structure of the data and could be used to proxy all sorts of UDP data.

Here is my code:

 -- Simple DNS Proxy  
 -- 20150923 by Andy Reischle  
 -- Blog: www.AReResearch.net  
 -- Vids: www.youtube.com/AReResearch  
 --  
 -- Uses googles dns server 8.8.8.8  
 -- change to whatever suits you  
 cu=net.createConnection(net.UDP,0)  
 cu:on("receive",function(cu,c)   
   -- print("Got a reply")  
   s:send(c)  
   end)  
 s=net.createServer(net.UDP)  
 s:on("receive",function(s,d)  
   -- print ("Got a request!")  
   cu:connect(53,"8.8.8.8")  
   cu:send(d)   
   end)  
 s:listen(53)  

The ESP-module is connected to a WiFi AP, of course. The IP address of the ESP module is 192.168.1.74 and will be different, depending on your home DHCP server.

I can query DNS information through the module now:
me@raspberrypi:~$ nslookup
> server 192.168.1.74
Default server: 192.168.1.74
Address: 192.168.1.74#53
> www.areresearch.net
Server:         192.168.1.74
Address:        192.168.1.74#53

Non-authoritative answer:
www.areresearch.net     canonical name = ghs.google.com.
ghs.google.com  canonical name = ghs.l.google.com.
Name:   ghs.l.google.com
Address: 173.194.65.121
>
The corresponding tcpdump also looks very clean:

me@raspberrypi:~$ sudo tcpdump -i eth0 port 53
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 65535 bytes
23:39:08.416736 IP noname.43027 > noname.domain: 56537+ A? www.areresearch.net. (37)
23:39:08.422009 IP noname.46122 > fritz.box.domain: 4560+ PTR? 58.1.168.192.in-addr.arpa. (43)
23:39:08.426857 IP fritz.box.domain > noname.46122: 4560* 1/0/0 PTR noname. (88)
23:39:08.428198 IP noname.45680 > fritz.box.domain: 56651+ PTR? 74.1.168.192.in-addr.arpa. (43)
23:39:08.430620 IP fritz.box.domain > noname.45680: 56651* 1/0/0 PTR noname. (88)
23:39:08.432969 IP noname.38314 > fritz.box.domain: 15794+ PTR? 1.1.168.192.in-addr.arpa. (42)
23:39:08.435987 IP fritz.box.domain > noname.38314: 15794* 1/0/0 PTR fritz.box. (89)
23:39:08.480105 IP noname.domain > noname.43027: 56537 3/0/0 CNAME ghs.google.com., CNAME ghs.l.google.com., A 173.194.65.121 (101)

While the rest of the project still requires a lot of tinkering, this little piece of the puzzle works nicely.


Tuesday, 22 September 2015

Changing the ASG Time Navigator TCP Port for MS-Exchange compatibility

At work we're in the process of upgrading from exchange 2007 to 2013. The windows guys got the job done with the usual little issues.
One unexpected issue occured when everything seemed to be ready to migrate the users to the new servers:
We were unable to install the Time Navigator Agent because TCP port 2525 was already in use by the exchange server as explained here.
We did not want to run into problems with future updates/patches on the Exchange server, so the TINA agent had to move over and use a different TCP Port.

That turned out to be easier than expected:
Time Navigator mostly relies on the services file of the operating system

Our backup server ist a CentOs machine, so the file is:
/etc/services

The  Exchange Servers are Windows servers, of course, so the services file is located at:
C:\WINDOWS\system32\drivers\etc

The backup server uses the catalog name as a service entry. You'll find something like:
yourcat          2525/tcp        # Time Navigator (yourcat)
yourcat-msg      2526/udp        # Time Navigator (yourcat)

On the Exchange Server, tina seems to be the generic name of the service
tina            2525/tcp        # Time Navigator (tina)
tina-msg        2526/udp        # Time Navigator (tina)

The default for the UDP Port is 2526 and there is absolutely nothing wrong with having the TCP protocol using the same port number. So TCP 2526 is the port of choice.

We changed:
/etc/services
to
yourcat          2526/tcp        # Time Navigator (yourcat)
yourcat-msg      2526/udp        # Time Navigator (yourcat)
(where "yourcat" is your catalog name)

and
C:\WINDOWS\system32\drivers\etc
to
tina            2526/tcp        # Time Navigator (tina)
tina-msg        2526/udp        # Time Navigator (tina)

That only did the trick partially. The web interface wouldn't connect to the catalog. So after some investigations, I found out that the web interface does not rely on the services file.

The ports are hard wired in:
/usr/Atempo/AtempoWebInterfaces/apache-tomcat/webapps/YOURCAT/WEB-INF/config/TinaConfiguration.xml

Change the ports in the <catalogs> and the <webRestore> section.
Restart Tina's Tomcat:
/usr/Atempo/AtempoWebInterfaces/rc_startup_aws.sh restart

brings the web interface back with it's full functionality.

On our Solaris and Linux clients, all it took was changing the entry in the services file, as well. Then restart the tina daemon. That's it.

Friday, 18 September 2015

New CaptiveIntraweb release V3 - ESP8266 / NodeMCU based captive portal

After a few hours of brushing things up, I released a new version of my CaptiveIntraweb on  my GitHub repo.
Apart from a few bug fixes, an all-new init.lua now does the Wifi Setup, compiles the LUA files and starts the servers. It also features an option to prevent the TCP and UDP servers from starting.

I did most of the development on my new, very user friendly V3 NodeMcu Lua WIFI Development Board I got directly from the PRC from Banggood. That board has 4MByte of flash memory. (Yes. I mean MBytes here, not MBit)

Looks a lot like a ESP-12E module on the board

If you don't know about CaptiveIntraweb, here is the original video from May 2015, although quite a bit of progress has been made since then.

With the new CaptiveIntraweb  loaded, the module startup now looks like this:
 ---------------------  
 Setting up WiFi AP...  
 Done.  
 ---------------------  
 Flash size is 4096 kBytes.  
 File system:  
  Total : 3360 kBytes  
  Used : 124 kBytes  
  Remain: 3236 kBytes  
 ---------------------  
 Compiling LUA files...  
 No need to compile   dns-liar.lua  
 No need to compile   server.lua  
 Compiling done.  
 ---------------------  
 Send some xxxx Keystrokes now to abort startup.  
 Will launch servers in 5 seconds...  
 > ---------------------  
 Starting HTTP Server  
 HTTP Server listening. Free Heap:    13080  
 Starting DNS Server  
 DNS Server listening. Free Heap:    10008  
 ---------------------  

Tuesday, 8 September 2015

Arduino and ESP8266 - part 2 - The web thermometer

In the previous video, I've shown various options to connect an ESP8266 module to an Arduino board. Now it is time to turn that into something (sortof) userful.

Watch this video to see how I build a web connected thermometer.



The program is a bit particular in that it wants AT-firmware 0.9.2.2 to run stable. I have tried the latest AT firmware but had no luck and didn't invenstigate any further. It's default baud rate of 9k6 is slow enough for a SoftwareSerial connection.

There are no surprises in the schematic diagram:



Here ist the code:


 /* ====== ESP8266 Web Thermometer Demo ======  
  * Print out temperature from TMP36 sensor  
  * on analog-in 3  
  * (Modified Aug 28, 2015 ARe)  
  * ==========================  
  *  
  * Change SSID and PASS to match your WiFi settings.  
  * The IP address is displayed to serial upon successful connection.  
  *  
  * modified by Andy Reischle (www.AReResearch.net)  
  * based on:  
  * Ray Wang @ Rayshobby LLC  
  * http://rayshobby.net/?p=9734  
  */  
 #define SSID "MYWIFI"   // change this to match your WiFi SSID  
 #define PASS "sorrywonttell" // change this to match your WiFi password  
 #define PORT "80"      // using port 80 by default  
 char buffer[BUFFER_SIZE];  
 // using Software Serial for connection to ESP  
 // Use the definitions below  
 #include <SoftwareSerial.h>  
 SoftwareSerial esp(10,11); // used pins 10, 11 for software serial   
 #define dbg Serial  
 // By default we are looking for OK\r\n  
 char OKrn[] = "OK\r\n";  
 byte wait_for_esp_response(int timeout, char* term=OKrn) {  
  unsigned long t=millis();  
  bool found=false;  
  int i=0;  
  int len=strlen(term);  
  // wait for at most timeout milliseconds  
  // or if OK\r\n is found  
  while(millis()<t+timeout) {  
   if(esp.available()) {  
    buffer[i++]=esp.read();  
    if(i>=len) {  
     if(strncmp(buffer+i-len, term, len)==0) {  
      found=true;  
      break;  
     }  
    }  
   }  
  }  
  buffer[i]=0;  
  dbg.print(buffer);  
  return found;  
 }  
 void setup() {  
  // assume esp8266 operates at 9600 baud rate  
  // change if necessary to match your modules' baud rate  
  esp.begin(9600);  
  dbg.begin(9600);  
  dbg.println("begin.");  
  setupWiFi();  
  // print device IP address  
  dbg.print("device ip addr:");  
  esp.println("AT+CIFSR");  
  wait_for_esp_response(1000);  
 }  
 bool read_till_eol() {  
  static int i=0;  
  if(esp.available()) {  
   buffer[i++]=esp.read();  
   if(i==BUFFER_SIZE) i=0;  
   if(i>1 && buffer[i-2]==13 && buffer[i-1]==10) {  
    buffer[i]=0;  
    i=0;  
    dbg.print(buffer);  
    return true;  
   }  
  }  
  return false;  
 }  
 void loop() {  
  int ch_id, packet_len;  
  char *pb;   
  if(read_till_eol()) {  
   if(strncmp(buffer, "+IPD,", 5)==0) {  
    // request: +IPD,ch,len:data  
    sscanf(buffer+5, "%d,%d", &ch_id, &packet_len);  
    if (packet_len > 0) {  
     // read serial until packet_len character received  
     // start from :  
     pb = buffer+5;  
     while(*pb!=':') pb++;  
     pb++;  
     if (strncmp(pb, "GET /", 5) == 0) {  
      wait_for_esp_response(1000);  
      dbg.println("-> serve homepage");  
      serve_homepage(ch_id);  
     }  
    }  
   }  
  }  
 }  
 void serve_homepage(int ch_id) {  
  String header = "HTTP/1.1 200 OK\r\nContent-Type: text/html\r\nConnection: close\r\nRefresh: 5\r\n";  
  String content="";  
 // read in TMP36 at Analog 3 port  
  int reading = analogRead(3);  
  float voltage = reading * 5.0;  
  voltage /= 1024.0;   
  float temperatureC = (voltage - 0.5) * 100 ;  
  Serial.print(temperatureC); Serial.println(" degrees C");  
  content += "Temperature is ";  
  content += temperatureC;  
  content += "C <br />\n";  
  header += "Content-Length:";  
  header += (int)(content.length());  
  header += "\r\n\r\n";  
  esp.print("AT+CIPSEND=");  
  esp.print(ch_id);  
  esp.print(",");  
  esp.println(header.length()+content.length());  
  if(wait_for_esp_response(2000, "> ")) {  
   esp.print(header);  
   esp.print(content);  
  } else {  
   esp.print("AT+CIPCLOSE=");  
   esp.println(ch_id);  
  }  
 }  
 void setupWiFi() {  
  // try empty AT command  
  esp.println("AT");  
  wait_for_esp_response(2000);  
  // set mode 1 (client)  
  esp.println("AT+CWMODE=1");  
  wait_for_esp_response(2000);   
  // reset WiFi module  
 /*  
  esp.print("AT+RST\r\n");  
  wait_for_esp_response(3000);  
  delay(5000);  
  */  
  // join AP  
  esp.print("AT+CWJAP=\"");  
  esp.print(SSID);  
  esp.print("\",\"");  
  esp.print(PASS);  
  esp.println("\"");  
  // this may take a while, so wait for 5 seconds  
  wait_for_esp_response(5000);  
  esp.println("AT+CIPSTO=30");   
  wait_for_esp_response(1000);  
  // start server  
  esp.println("AT+CIPMUX=1");  
  wait_for_esp_response(1000);  
  esp.print("AT+CIPSERVER=1,"); // turn on TCP service  
  esp.println(PORT);  
  wait_for_esp_response(1000);  
 }  

I have made two modifications to the code:

  • To keep the Arduino's original debugging facility, I kept the hardware UART for that purpose and defined a "software serial" port on pins 10 and 11. The suggested firmware version defaults to 9600 baud, so speed is not an issue.
  • The other change is that I nicked a few lines of code from Adafruit to replace the original output

Thursday, 27 August 2015

Level converters - How to safely connect an Arduino to a ESP8266 module



This is the companion blog article to my video on YouTube.
A second part with a practical application will follow.

Option 1: Bidirectional N-type Mosfet level shifter

This is my favourite "no worries" solution. These modules provide 4 bidirectional lines. Enough for most of my little projects and more than enough for a serial connection between an Arduino Uno and a ESP-01 module.


These level converters can be obtained for cheap from here.

Option 2: Voltage follower

Not bidirectional but also a very clean solution is this voltage follower. The output level follows the input signal but only to the point when it reaches the supply voltage. I.e. 3V in this case.

The popular 2n2222a transistors will serve the same purpose and can be bought here.


Option 3: Zener clamp

This is a great option if you want a reliable solution with a minimum of components. The Zener diode clamps down the voltage on the 3V side of the resistor. (Output signal shown in green here.)

Here is a link to a set of zener diodes.



Option 4; Voltage divider

By far the cheapest option for a quick hack. The classic voltage divider. It does not provide any additional protection against voltage surges, but where those are not to be expected, you'll easily get away with that simplest of circuits.
Resistors are dirt cheap and you should always have some available. Here is a very basic set.

Tuesday, 18 August 2015

Problems on windows 2008R2 production server following August 2015 patches

As a followup to this post, two of the August 2015 updates apparently fix the same issue, causing problems on one of our servers.

While in July the patches were:
KB3067505 and KB3057154

This time it is:
KB3060716 and  KB3071756

With these patches installed, a hot folder mechanism refused to work more than  once. The actual mechanism behind it still remains in the dark at the moment. Looking at the security bulletins, the problem appears to center around a privilege escalation bug.

Removing these two patches brought everything back to normal.


Monday, 10 August 2015

Windows 10 (nearly) broke my Intel 4 Series Express graphics adapter


After an otherwise smooth transition from windows 8.1 to  Win 10 on my Dell E6400, OpenGL applications stopped working.

Stellarium relies on OpenGL
My graphics adapter apparently does not support openGL any more. It is an:
"Intel 4 series Express Chipset Family" adapter that has gone out of active support for quite a while.

After some failures, I found a surprisingly simple solution:
This Win7 driver from the Intel download center installed without any issues and brought OpenGL back at an (for this system) acceptable speed.
OpenGL is back - and so are the stars.

While a slightly newer installer (Win7Vista_64_151719.zip) refused to run on Win10, this one (Win7Vista_64_151717bb.zip) did.
I will probably have to prevent windows from updating that driver in the future.

20150824 Adendum:
The sound output of the docking station wouldn't work either. Replacong the IDT audio driver with the standard high-definition audio driver fixed that.