This is what you get when you blutac a GPS receiver to your window when going for a domestic flight. Altitude is colour coded.
Taking off from Sydney:
Here, speed is colour coded:
You can download the YMML-YSSY trip: YMML-YSSY
Shows areas of Australia that have been crawled for sites (tiles of different sizes are difference colours). Appears most population centres are covered, but much is still left to crawl.
This image shows all 2057 cached sites that are labelled “Telstra customer”:
The image shows the position of all Radio Frequency (RF) transmitter sites cached in the RFMap database so far. Click the image for a full-size view. Click here to visit RFMap.
The first iteration of the Australian Geographical RadioFrequency Map is live!
I have multiple web servers behind one IP address and use port forwarding to each allowing external access. The main web server is externally accessible on port 80, but the others require manual port selection in a URL. This is annoying to say the least, and potentially problematic if a surfer is restricted only to port 80.
The solution is to use Squid in reverse proxy, or web/http accelerator, mode. The turns Squid on its head, serving requests from the outside and fetching content from internal servers. In effect, it listens on external port 80 and connects to the correct internal web server depending on the Host header in the HTTP request.
Depending on your requirements there are a few issues to take into account. In my particular case, I did not want Squid to perform any caching – it is purely for routing external requests. The other major requirement is that logging of external IP addresses should still work in each web server’s access log. This is easily achieved with a bit of additional configuration.
Your complete squid.conf should be made up of the following:
http_port 8080 vhost
I port forward external port 80 to internal port 8080, as there is already a web server running on port 80 on the same machine (it is generally not a good idea to have Squid and Apache on the same machine, but I’m not caching to disk and my site isn’t that popular). An additional ‘default_site=<host>’ option can be added if you want to handle requests that do not specify a particular Host.
cache_peer <Web server IP> parent 80 0 no-query originserver no-digest login=PASS name=<Any name for later reference>
‘no-query’ turns of ICP requests, ‘originserver’ tells Squid this is a web server, ‘no-digest’ turns off Squid’s annoying requests to ’squid-internal-periodic’ on each web server (which would never be fulfilled), ‘login=PASS’ allows basic authentication to be passed to the web servers (it tells Squid to trust them, and not strip the Authorization or WWW-Authenticate headers from requests), and ‘name’ is just a name we use later (I use ‘<hostname>_<server type>’).
Then for each Host/domain you wish to forward:
acl <acl name> dstdomain <Host/domain> http_access allow <acl name> cache_peer_access <cache_peer name> allow <acl name> [Remaining acl's, if any.] cache_peer_access <cache_peer name> deny all
A nice addition for testing is:
acl LocalWWW dstdomain <internal test hostname> acl localnet src 192.168.0.0/16 # RFC1918 possible internal network http_access allow LocalWWW localnet [Place amongst cache_peer_access lines for default web server, before 'deny all' line.] cache_peer_access <cache_peer name of default server> allow LocalWWW
This allows you to test Squid on your internal network by requesting a page from an internal hostname (e.g. if you run an internal DNS server). This is obviously inaccessible from the outside world.
To tighten things up, also add:
acl localhost src 127.0.0.1/32 acl manager proto cache_object acl Safe_ports port 80 # http acl CONNECT method CONNECT http_access allow manager localhost http_access deny manager http_access deny !Safe_ports http_access deny CONNECT http_access deny all
Make sure the last line is the last of all the ‘http_access’ lines!
This is where the tricks begin:
X-Forwarded-For is the de facto HTTP request Header for notifying upstream servers that the request has been forwarded by a proxy server. This is the key to enabling logging of external IP addresses by the web server: instead of logging the IP address of the client connection (which will always be that of Squid’s machine), it will use the value in this header. This can be easily done in Apache by:
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined_forward_for
There is one problem: if the request has already come from a proxy and adds X-Forward-For to the request header before arriving at this Squid server, this Squid instance will add this machine’s IP to the header (as per the spec), which will result in a list of IP addresses. If the above ‘LogFormat’ is used as-is, log entries will be invalid (causing log analysers to reject such lines). This is not what we want. To work around this, add the following to squid.conf:
header_replace X-Forwarded-For
This instructs Squid to replace an incoming request’s X-Forward-For header with nothing (i.e. remove it). Luckily, this is done before Squid adds its own entry to X-Forward-For, which means there will always be only one IP address in the header when the request is forwarded on.
(If you wish to skip the remainder of the Squid configuration tips and proceed directly to the rest of logging in Apache, click here.)
Next, you probably don’t want Via, X-Cache and X-Squid headers to be sent back to clients, as you would like Squid’s presence to be (mostly) invisible to the outside world. You can add the following to instruct Squid not to add such headers when replying to external requests (they are still added for internal network requests, which can help with debugging):
via off reply_header_access X-Cache-Lookup deny !localnet reply_header_access X-Squid-Error deny !localnet reply_header_access X-Cache deny !localnet
To avoid caching anything (just act as a request router):
cache deny all
To conserve memory:
memory_pools off
If you are already running squid on the same machine in normal mode (i.e. serving internal clients), then specify a different SNMP port (if you’re using SNMP on the original instance):
acl snmppublic snmp_community public snmp_access allow snmppublic localnet snmp_access deny all snmp_port 3402 # +1 from default
If you use ICP and have an existing instance, set a different port:
icp_port 3131 # +1 from default
To speed up restarts while testing:
shutdown_lifetime 0 seconds
Don’t forget your log:
access_log /var/log/squid3/access.log squid
And finally:
cache_mgr <spam-safe email address, or something made-up if you don't wish to be contacted> cachemgr_passwd <password>
That’s it for Squid. Now to Apache:
In addition to the custom ‘LogFormat’ specified above, you can add this to a new file in the ‘conf.d’ directory:
SetEnvIf Remote_Addr "^127\.0\.0\.1$" local_network SetEnvIf Remote_Addr "^192\.168\." local_network SetEnvIf X-Forwarded-For "^$" normal_request SetEnvIf X-Forwarded-For ".+" via_accel
Then in each virtual host (’VirtualHost’) add:
RewriteEngine On
RewriteCond %{ENV:local_network} =1
RewriteCond %{ENV:via_accel} !=1
RewriteRule .* - [env=local_request:1]
These directives will set the ‘local_request’ variable whenever a direct request is made to the webserver on the internal network. This is good for restricting directory/page access to internal clients, but not requests forwarded through Squid (which would appear as a local network client if only using IP address restriction). An example is:
<Location /blah> Order Allow,Deny Allow from env=local_request </Location>
Now the grand finale: logging. Just use the following:
CustomLog /var/log/apache2/access.log combined_forward_for env=via_accel CustomLog /var/log/apache2/access.log combined env=normal_request
The same file can be used over multiple ‘CustomLog’ lines, and the appropriate line is chosen depending on whether the request is direct or forwarded through Squid. Either way, the origin IP address is logged.
If you use IIS, then this ISAPI DLL is for you. It will detect the X-Forwarded-For header and change the request address server variable to the X-Forwarded-For value (I do not know how it handles the IP address list problem). Remember to have it execute first in the filter list.
A good place to visit when getting started with Squid configuration is their wiki. SNMP information can be found here.
This is a good page on understanding the refresh_pattern directive.
As far as Windows-based decent free screen-recording software goes, CamStudio is it. When installing, make sure you also download and install the CamStudio Lossless Codec (CSCD)! There are a couple of problems that become apparent when actually using captured video in other video processing apps (e.g. Media Player Classic (MPC), Adobe Premiere and VirtualDub). The following list assumes video is compressed with CSCD, unless otherwise noted.
I recommend the following modifications to resolve the above problems, respectively:
Having recently upgraded to Visual Studio 2008, my lackluster experience with developing and debugging native code in the ‘improved’ IDE has left me looking (and I am still) for hotfixes that might patch various painful problems. In particular, the IDE suffers major performance problems when debugging large C++ projects that (believe it or not) have breakpoints set. With a little experimentation I discovered that actually have any breakpoints at all causes a massive performance hit when single-stepping through one’s code, whether or not they are disabled. This is not a good thing for a debugger and its GUI, which is a real shame since Visual Studio 2005 was not plagued by any such issues.
This leads me to the following idea: why not keep using the Visual Studio 2005 IDE for development and debugging, but make it use the VC 2008 compiler and newer libraries (i.e. MFC 9 and the VC++ 2008 redistributables)? This will mean that Visual Studio 2005 will build your projects with the newer compiler and link to the latest versions of the runtime/MFC/etc. Since the 2005 debugger will work for either version (i.e. 2005- or 2008-compiled code), one can safely revert back to Visual Studio 2005 for such simple activities as single-stepping (phew!). I must point out that one good think Microsoft did for Visual Studio 2008 was dramatically improve their performance analysis tools – this would be the only reason to use the 2008 IDE: the newer profiling tools/GUI.
The first thing to do is to identify the full path of Visual Studio 2008’s ‘VC’ directory, and where version ‘6.0A’ of the Platform SDK resides. By default they are (respectively):
C:\Program Files\Microsoft Visual Studio 9.0\VC
C:\Program Files\Microsoft SDKs\Windows\v6.0A
Next you need to create a property sheet that contains the necessary information for a VS 2005 C/C++ project to use the 2008 headers and libraries.
Create a new property sheet file manuall (e.g. “VS9.vsprops”) and insert the following, adjusting the paths where necessary:
<?xml version="1.0" encoding="Windows-1252"?> <VisualStudioPropertySheet ProjectType="Visual C++" Version="8.00" Name="VS9" > <Tool Name="VCCLCompilerTool" AdditionalIncludeDirectories=""C:\Program Files\Microsoft Visual Studio 9.0\VC\include";"C:\Program Files\Microsoft Visual Studio 9.0\VC\atlmfc\include";"C:\Program Files\Microsoft SDKs\Windows\v6.0A"" /> <Tool Name="VCLinkerTool" AdditionalLibraryDirectories=""C:\Program Files\Microsoft Visual Studio 9.0\VC\lib";"C:\Program Files\Microsoft Visual Studio 9.0\VC\atlmfc\lib";"C:\Program Files\Microsoft SDKs\Windows\v6.0A\Lib";"C:\Program Files\Microsoft Visual Studio 9.0"" /> <Tool Name="VCResourceCompilerTool" AdditionalIncludeDirectories=""C:\Program Files\Microsoft Visual Studio 9.0\VC\include";"C:\Program Files\Microsoft Visual Studio 9.0\VC\atlmfc\include";"C:\Program Files\Microsoft SDKs\Windows\v6.0A\Include"" IgnoreStandardIncludePath="true" /> </VisualStudioPropertySheet>
Then in Visual Studio 2005, open the Property Manager (via the View menu), and add this new file. Make sure in the Project Settings, under General, the property “Inherited Project Property Sheets” includes the new file.
The last trick is to add the following paths to the “Executable Files” list in VS 2005 under Tools -> Options -> Projects and Solutions -> VC++ Directories:
C:\Program Files\Microsoft SDKs\Windows\v6.0A\bin
C:\Program Files\Microsoft Visual Studio 9.0\VC\bin
C:\Program Files\Microsoft Visual Studio 9.0\Common7\IDE
The first is for RC.exe, the second is the compiler and the third is for the linker to find DLLs for manipulating debug information (e.g. mspdb80.dll).
NOTE: They don’t have to be added at the very top of the list (e.g. the NASM directory might be at the top) but they must come before the VS 2005-versions of the same directories. That is, they should come before the following since they are supposed to override them (that said, you shouldn’t have to remove any entires if the order is correct):
$(VCInstallDir)bin
$(VSInstallDir)Common7\Tools\bin
$(VSInstallDir)Common7\ide
Also, note: once you put these new paths in, ALL projects compiled under VS 2005 will use the VC 2008 compiler and tools. Unfortunately there is no clean way to override the compiler/tool paths using the VSProps method above. I tried creating a user-defined macro in the VSProps named ‘VSInstallDir’ to override the in-built one, but it had no effect. Although I haven’t tested it, there shouldn’t be a problem if you use the new compiler on VS 2005 projects without the inherited VSProps – you’ll just have a newer compiler linking to MFC 8 and the VC 2005 runtime, which shouldn’t (hopefully) cause any dramas!
Latest Content Wiki Gallery© 2007 Balint's Blog. Powered by Wordpress wearing MyJournal Hypereal by 
BlogoSquare
There are currently 37 posts extending over 0 categories with loads upto 8 comments with valid XHTML and CSS.