[TriLUG] seeking WGET assistance, maybe LYNX is the answer ??

[Aaron S. Joyner]

Turnpike Man wrote:

>Interestingly enough, their little HTTPGET.exe proggy is somewhat of a
>missnomer.  I don't think it is truely doing an HTTP GET, opposite of HTTP
>POST.  However, it kinda is, because the "iServer" embedded web device is what
>is supposed to respond...
>  
>
It sounds like you're not entirely sure what is supposed to go back and 
forth between the device...  You need to be sure of that, before you try 
implementing a solution.

>Tom put in some interesting python code, again, python, perl, and anything
>outside of HTML, PHP, SQL are greek to me.  However, I'm still open to such
>options.
>  
>
Given this, what I'm about to suggest may be beyond your *current* 
abilities, but it's not terribly difficult.  With a little patience, the 
right tools, and friendly help from the list, anyone who can operate a 
web browser is capable of working through this.

>Latest news is... the "continuous mode" in which I can connect via telnet and
>get a data stream of temp, humidity, dewpoint, time, temp, humidity, dewpoint,
>time, etc... has lost the time part, only spewing t, h, d now.  So I'm working
>with those folks to get that right.
>
Mmm... I don't think you can fix that.  Presuming it dumps the wrong 
output when you follow their normally-prescribed instructions, you've 
got to go to them to get a resolution on why it's not properly tracking 
the time.  Perhaps it's supposed to connect to an NTP server that it 
can't?  Perhaps you need to set the time in some fashion before it can 
output it on it's own?

>In the end, I guess I only need collect my
>computer timestamp and not worry about the device timestamp when using the
>streamed data.
>
This is of course an entirely acceptable work-around, presuming your 
monitoring station has a clock (and unless it's a tiny embedded device, 
you can probably bet it does).

>There is also "command mode" in which I can connect via telnet
>and 'paste' the "*SRT" command to get temp and timestamp back.  I dunno why
>paste must be used instead of typing the command, but that's telnet for ya I
>guess.  However, now this only reports temp and no timestamp either.
>  
>
I'd guess they need to get all 4 of those characters in a single packet, 
and typing them in will send them in 4 separate packets to the remote 
host (which brings me one step closer to my suggestion)...

>Fortunately timestamping on the internal logging of the device is fine, it's
>just not spewing it to me when I send it telnet or httpget.exe commands
>anymore.
>  
>
That shoots my earlier NTP idea in the foot.  Have to get them to fix 
their firmware if it's not doing what it's supposed to be.  Read on for 
things you can fix, but that's not one of them.  :)

>Found out my WGET is 1.8.2-15.3 in FC1, not recent enough to include
>--post-data=STRING option.
>  
>
That's certainly a problem.  You might consider upgrading wget, or 
looking a "curl", as I suggested in a previous post.

>Also, the -r means NO HTTP headers in their httpget.exe proggy... it's a raw
>mode of connection that I think support says won't work if you include any HTTP
>header communication.
>  
>
Here we are back at the original problem.  It appears that you're not 
really sure what httpget.exe is doing, or exactly what communication is 
transpiring.  If you have the capability to run httpget.exe for testing 
- it's time to break out a packet sniffer and do some testing.  If 
you're going to be in Windows, you can download Ethereal, and sniff your 
packets right there on the same machine running httpget.exe.  That's 
probably the easiest solution.  To keep the situation simple, you'll 
want to keep the amount of TCP/IP "chatter" on the machine to a minimum 
(don't have a web-browser open, don't stream NPR in the background while 
you're doing this, etc.).  Yes, you can do away with all that chatter 
with a simple Ethereal filter, but we're trying to keep this easy.  :)  
For a few sample filter tricks, check the footnote at the bottom of this 
message.

So, download and install Ethereal.  Start up the main program.  Click 
Capture from the menu options at the top of the screen, and then choose 
Start.  Choose your interface from the drop down menu at the top of the 
screen (you'll need to select the Ethernet interface that the device is 
connected on - if there are no interfaces available, make sure you're an 
administrator and then try again).  Once you've selected the correct 
interface, click "Ok".  Now go and perform your "query" with 
httpget.exe.  Try and keep it short, only do it once at first.  Then 
come back to Ethereal, and stop the capture.

Now you will have a big long list of packets that were passed back and 
forth between you and the remote machine.  For the simplest view of what 
needs to go back and forth, pick one with a destination address of the 
remote host.  Right click on it.  Select "Follow TCP Stream", which 
should be the first option.  This will open up another window, and show 
you the text that was passed back and forth between your host, and the 
remote host.  Traffic sent by you will be in Pink, traffic sent from the 
remote host will be Blue.  You should be able to get a rough idea of the 
data traveling back and forth between you and the remote host.

Once you have that data, you can post the text traffic, or preferably 
the entire libpcap dump (File->Save As) here and we can give you a hand 
at dissecting it.  With that information, it's much easier to tell you 
what command, with what options, will give you the results you desire.

>still pluggin away,
>  
>
Don't stop plugging.  Reverse engineering / protocol decoding / low 
level hacking of this nature can be both very fun and very rewarding, if 
you approach it with the right mindset.  But be careful, it's very easy 
to learn a lot along the way.  :)

Aaron S. Joyner

Footnote:  To filter what packets Ethereal will record, here are two 
quick-and-dirty methods.  For each of them, you can put the filter 
string in the "Capture Filter:" section of the "Capture Options" screen 
(opened by choosing Capture -> Start).  To capture all TCP/IP packets 
corresponding to a single host, use "host 1.2.3.4" where 1.2.3.4 is the 
IP address of the machine.  You could also use the machine's DNS name if 
you so desired.  Note though, that this method doesn't capture the ARP 
packets associated with that machine, which can sometimes be useful for 
certain types of debugging.  To do that, you need to know the MAC 
address of the device.  There are lots of ways to determine the ARP 
address, but one sure-fire way is to do a capture based on the host 
address, and then look at the center pane in Ethereal to find the MAC 
address.  Once you have determined the MAC address, you can  use a 
syntax like this to capture all packets that go to and from that 
physical device: "ether host 00:01:03:E9:4A:1C".  This way, you can keep 
streaming NPR while you do your captures.  :)