HTTPS through CherryPy webserver standalone

In trying to get my standalone CherryPy webserver to serve over https, I can across the following CherryPy documentation: http://www.stderr.org/doc/cherrypy-doc/html/howto/node10.html. It stated:

Once you have PyOpenSSL installed, all you have to do is add 2 lines in your CherryPy config file, in the server section:

sslKeyFile=/path/to/ssl/key/file
sslCertificateFile=/path/to/ssl/certificate/file

And that's it ! 

Not so much. I tried this, and then tried to connect over https. The result?

SSL received a record that exceeded the maximum permissible length

I believe those lines are for configuration with use of Apache, not the standalone server. To get the standalone server working, I had to set server variables as follows (substitute your own relative key/cert/port values):

    cherrypy.server.socket_port = 443
    cherrypy.server.ssl_certificate = 'ssl/server.crt'
    cherrypy.server.ssl_private_key = 'ssl/rsa.key'

VBA Excel - Formatting a column of numbers to a given precision, stripping out the decimal place, and adding leading zeros to satisfy a minimum length

'Return the number dNumber increased to the power iExponent
Function Pow(ByVal dNumber As Double, ByVal iExponent As Integer) As Double
    Pow = 1
    
    Dim iCounter As Integer
    
    For iCounter = 1 To iExponent
        Pow = Pow * dNumber
    Next iCounter
End Function

'Convert the values in the active column to the given precision, with no decimal place, filled with leading zeros to satisfy a minimum length
Sub ConvertColToTextNoDecimal(ByVal iPrecision As Integer, ByVal iMinimumLength As Integer)
    Dim row As Long
    Dim col As Long
    
    col = Selection.Column 'We are going to process whatever column is currently selected
    
    Dim val As Variant
    
    For row = 2 To ActiveSheet.UsedRange.Rows.Count 'Assume we are processing from below a header row to the last used row in the selected sheet
        val = ActiveSheet.Cells(row, col).Value 'Get the current value
        
        On Error GoTo NextRow
        val = CStr(Round(CDbl(val) * Pow(10, iPrecision), 0)) 'Round to the given precision, then shift off the decimal place
        
        While Len(val) < iMinimumLength 'If the value we have isn't long enough:
            val = "0" & val             'Prepend a zero
        Wend
        
        ActiveSheet.Cells(row, col).FormulaR1C1 = "'" & val 'Mark this value as text, so we don't lose any leading zeros
NextRow:
        On Error GoTo 0
    Next row
    
End Sub

Example usage:

'Round to 3 decimal places, remove the decimal point
Sub ConvertColTo3Dec()
    ConvertColToTextNoDecimal 3, 0
End 

'Round to 2 decimal places, remove the decimal point
Sub ConvertColTo2Dec()
    ConvertColToTextNoDecimal 2, 0
End Sub

'Round to 1 decimal place, remove the decimal point
Sub ConvertColTo1Dec()
    ConvertColToTextNoDecimal 1, 0
End Sub

'Round to 0 decimal places
Sub ConvertColTo0Dec()
    ConvertColToTextNoDecimal 0, 0
End Sub

'No decimal place changes, but force enough leading zeros for 3 digits
Sub ConvertColTo0Filled3Long()
    ConvertColToTextNoDecimal 0, 3
End Sub

Django: AttributeError: 'module' object has no attribute 'JSONEncoder'

Running a Django syncdb, I've encountered a problem: AttributeError: 'module' object has no attribute 'JSONEncoder'.

$ python ./manage.py syncdb
Traceback (most recent call last):
  File "./manage.py", line 11, in ?
    execute_manager(settings)
  File "/usr/lib/python2.4/site-packages/django/core/management/__init__.py", line 347, in execute_manager
    utility.execute()
  File "/usr/lib/python2.4/site-packages/django/core/management/__init__.py", line 295, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/usr/lib/python2.4/site-packages/django/core/management/base.py", line 195, in run_from_argv
    self.execute(*args, **options.__dict__)
  File "/usr/lib/python2.4/site-packages/django/core/management/base.py", line 222, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python2.4/site-packages/django/core/management/base.py", line 351, in handle
    return self.handle_noargs(**options)
  File "/usr/lib/python2.4/site-packages/django/core/management/commands/syncdb.py", line 149, in handle_noargs
    call_command('loaddata', 'initial_data', verbosity=verbosity)
  File "/usr/lib/python2.4/site-packages/django/core/management/__init__.py", line 158, in call_command
    return klass.execute(*args, **options)
  File "/usr/lib/python2.4/site-packages/django/core/management/base.py", line 222, in execute
    output = self.handle(*args, **options)
  File "/usr/lib/python2.4/site-packages/django/core/management/commands/loaddata.py", line 91, in handle
    formats = serializers.get_public_serializer_formats()
  File "/usr/lib/python2.4/site-packages/django/core/serializers/__init__.py", line 72, in get_public_serializer_formats
    _load_serializers()
  File "/usr/lib/python2.4/site-packages/django/core/serializers/__init__.py", line 108, in _load_serializers
    register_serializer(format, BUILTIN_SERIALIZERS[format], serializers)
  File "/usr/lib/python2.4/site-packages/django/core/serializers/__init__.py", line 50, in register_serializer
    module = __import__(serializer_module, {}, {}, [''])
  File "/usr/lib/python2.4/site-packages/django/core/serializers/json.py", line 44, in ?
    class DjangoJSONEncoder(simplejson.JSONEncoder):
AttributeError: 'module' object has no attribute 'JSONEncoder'

Googling says that the problem is that I have python-json installed, and that I need to have python-simplejson installed too. Except that I already do have that installed. So, when running a syncdb, just move json.py in the Python site-packages directory aside for a moment, and everything should run fine.

sslerror: (8, 'EOF occurred in violation of protocol')

I have a program in the field that performs SOAP calls to a server over HTTPS. The program works fine at most places. However, for one customer, the relevant call gives an error of

sslerror: (8, 'EOF occurred in violation of protocol')

I started by taking a look at the stack trace on this guy:

Traceback (most recent call last):
  File "/roadware/tire/custom/rt/dcpr", line 257, in main
    products = q.getPrices(dealer=opts.dealer, customer_id=opts.customer, document_date=opts.date, pricing_type_id="1", products=opts.products)
  File "/roadware/tire/custom/rt/dcpr", line 149, in getPrices
    response = self.call(fullaction, gpelem, headerelem)
  File "/usr/lib/python2.3/site-packages/elementsoap/ElementSOAP.py", line 209, in call
    parser=namespace_parse
  File "/usr/lib/python2.3/site-packages/elementsoap/HTTPClient.py", line 147, in do_request
    h.endheaders()
  File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/httplib.py", line 712, in endheaders
    self._send_output()
  File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/httplib.py", line 597, in _send_output
    self.send(msg)
  File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/httplib.py", line 564, in send
    self.connect()
  File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/httplib.py", line 985, in connect
    ssl = socket.ssl(sock, self.key_file, self.cert_file)
  File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/socket.py", line 73, in ssl
    return _realssl(sock, keyfile, certfile)
  sslerror: (8, 'EOF occurred in violation of protocol')

Googling around for this problem suggests that proxies can result in this problem, which is likely caused by the HTTP header send being terminated prematurely. A few people suggested using a different SSL library (M2Crypto), but my solution makes use of ElementSOAP, and I'd really rather not have to mess with that much of the underlying code. So I took a look at the source of these Python packages:

In HTTPClient.py:

h.putrequest(method, path)
h.putheader("User-Agent", self.user_agent)
h.putheader("Host", self.host)
h.putheader("Content-Type", content_type)
h.putheader("Content-Length", str(len(body)))
if extra_headers:
    for key, value in extra_headers:
 h.putheader(key, value)
h.endheaders()

h.send(body)

# fetch the reply
errcode, errmsg, headers = h.getreply()

It looks like the header send is blowing up before the body of the SOAP message goes--we're not going to be able to just suppress the exception and take our response from the server.

In ElementSOAP.py:

# call the server
try:
    response = self.__client.do_request(
 ET.tostring(envelope),
 extra_headers=[("SOAPAction", action)],
 parser=namespace_parse
 )
except HTTPError, v:
    if v[0] == 500:
 # might be a SOAP fault
 response = namespace_parse(v[3])

Well, this is interesting. I'm not an expert on proxies or HTTP headers, but I see that SOAPAction extra header getting sent, and I can envision a proxy stripping out that header before sending it on to it's final destination. Result? Headers in violation of protocol (according to IIS)? Maybe. However, the customer claims there is no proxy.

My next thought? This post suggests changing the defaultHttpsTransport in client.py from
httplib.HTTPConnection to M2Crypto.httpslib.HTTPSConnection. Unfortunately, I'm working on SCO Openserver, so I don't know if I'll be able to build M2Crypto for this to work.

Instead, I tried to investigate the problem further. On the non-functioning machine, I just tried to access the URL I am targeting using curl. The result?

curl: (1) libcurl was built with SSL disabled, https: not supported!

Really? Well, just be be sure this would work at all, I tried it on a machine that didn't have the problem.

curl: (60) error setting certificate verify locations:
  CAfile: /usr/share/curl/curl-ca-bundle.crt
  CApath: none

More details here: http://curl.haxx.se/docs/sslcerts.html

curl performs SSL certificate verification by default, using a "bundle"
 of Certificate Authority (CA) public keys (CA certs). The default
 bundle is named curl-ca-bundle.crt; you can specify an alternate file
 using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
 the bundle, the certificate verification probably failed due to a
 problem with the certificate (it might be expired, or the name might
 not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
 the -k (or --insecure) option.

Coincidence? Or are we onto something? I find another machine that hasn't been running this program, and try the curl command.

curl: (1) libcurl was built with SSL disabled, https: not supported!

And the program gives sslerror: (8, 'EOF occurred in violation of protocol') on this system.

Maybe SSL isn't built the same on the failing machines?

One more time, but on a different machine that is working. This time, the curl command works.

It turns out the failing systems are all running SCO Openserver 5.0.6, while the working systems are running SCO Openserver 5.0.7. Maybe there's a problem with the SSL library installed on 5.0.6? Or the Python installation on those? Or something being used didn't have SSL compiled in, like curl? So maybe the python wrapper calling the SSL library isn't getting what it should be getting out of the library, and then is sending something wrong to the server instead of notifying that something went wrong earlier.

At this point, we've decided we are going to require 5.0.7 for this program, so I'm no longer actively researching a solution.

Code Syntax Highlighting on Blogger

Summary:

If you want your code formatted in a Blogger post, just add the following two lines to your template, before the </body> tag.

<script src='http://easy-blog-code-syntax-highlighting.googlecode.com/svn/trunk/ebcsh.js' type='text/javascript'/>

<link href='http://www.geocities.com/easyblogcodesyntax/ebcsh.css' rel='stylesheet' type='text/css'/>

Then, just wrap any code in <pre> tags, make sure you add the name="code" attribute, and select the language you are formatting for in the class:

<pre name="code" class="brush: c-sharp;">
function test() : String
{
 return 10;
}
</pre>

becomes

function test() : String
{
 return 10;
}

Explanation:

The first order of business for this blog was getting code syntax highlighting working. Not too difficult, there's a utility called SyntaxHighlighter that will do this in JavaScript, and instructions here and here on making it work in Blogger. Unfortunately, this still required hosting the Syntax Highlighter files somewhere, and adding some custom code from one of the above posts, or else Blogger adds visible <br/> tags to the line breaks in the code.

So, I combined all the relevant JavaScript from the Syntax Highlighter project (built against 2.0.287), added the necessary code to remove the <br/> tags, and put that up on a Google code project here. Then I just linked to that file (ebcsh.js) directly out of the svn trunk.

The css required a little more work, as Firefox refused to load css served from the Google Code's svn trunk. Google served the file with a MIME type of "text/plain", instead of "text/css", so I dialed up 1997 for some swanky free Geocities hosting.