Monday, February 23, 2009

HTTPS through CherryPy webserver standalone

In trying to get my standalone CherryPy webserver to serve over https, I can across the following CherryPy documentation: It stated:

Once you have PyOpenSSL installed, all you have to do is add 2 lines in your CherryPy config file, in the server section:


And that's it !

Not so much. I tried this, and then tried to connect over https. The result?

SSL received a record that exceeded the maximum permissible length

I believe those lines are for configuration with use of Apache, not the standalone server. To get the standalone server working, I had to set server variables as follows (substitute your own relative key/cert/port values):

cherrypy.server.socket_port = 443
cherrypy.server.ssl_certificate = 'ssl/server.crt'
cherrypy.server.ssl_private_key = 'ssl/rsa.key'

Wednesday, February 18, 2009

VBA Excel - Formatting a column of numbers to a given precision, stripping out the decimal place, and adding leading zeros to satisfy a minimum length

'Return the number dNumber increased to the power iExponent
Function Pow(ByVal dNumber As Double, ByVal iExponent As Integer) As Double
Pow = 1

Dim iCounter As Integer

For iCounter = 1 To iExponent
Pow = Pow * dNumber
Next iCounter
End Function

'Convert the values in the active column to the given precision, with no decimal place, filled with leading zeros to satisfy a minimum length
Sub ConvertColToTextNoDecimal(ByVal iPrecision As Integer, ByVal iMinimumLength As Integer)
Dim row As Long
Dim col As Long

col = Selection.Column 'We are going to process whatever column is currently selected

Dim val As Variant

For row = 2 To ActiveSheet.UsedRange.Rows.Count 'Assume we are processing from below a header row to the last used row in the selected sheet
val = ActiveSheet.Cells(row, col).Value 'Get the current value

On Error GoTo NextRow
val = CStr(Round(CDbl(val) * Pow(10, iPrecision), 0)) 'Round to the given precision, then shift off the decimal place

While Len(val) < iMinimumLength 'If the value we have isn't long enough:
val = "0" & val 'Prepend a zero

ActiveSheet.Cells(row, col).FormulaR1C1 = "'" & val 'Mark this value as text, so we don't lose any leading zeros
On Error GoTo 0
Next row

End Sub

Example usage:

'Round to 3 decimal places, remove the decimal point
Sub ConvertColTo3Dec()
ConvertColToTextNoDecimal 3, 0

'Round to 2 decimal places, remove the decimal point
Sub ConvertColTo2Dec()
ConvertColToTextNoDecimal 2, 0
End Sub

'Round to 1 decimal place, remove the decimal point
Sub ConvertColTo1Dec()
ConvertColToTextNoDecimal 1, 0
End Sub

'Round to 0 decimal places
Sub ConvertColTo0Dec()
ConvertColToTextNoDecimal 0, 0
End Sub

'No decimal place changes, but force enough leading zeros for 3 digits
Sub ConvertColTo0Filled3Long()
ConvertColToTextNoDecimal 0, 3
End Sub

Friday, February 13, 2009

Django: AttributeError: 'module' object has no attribute 'JSONEncoder'

Running a Django syncdb, I've encountered a problem: AttributeError: 'module' object has no attribute 'JSONEncoder'.

$ python ./ syncdb
Traceback (most recent call last):
File "./", line 11, in ?
File "/usr/lib/python2.4/site-packages/django/core/management/", line 347, in execute_manager
File "/usr/lib/python2.4/site-packages/django/core/management/", line 295, in execute
File "/usr/lib/python2.4/site-packages/django/core/management/", line 195, in run_from_argv
self.execute(*args, **options.__dict__)
File "/usr/lib/python2.4/site-packages/django/core/management/", line 222, in execute
output = self.handle(*args, **options)
File "/usr/lib/python2.4/site-packages/django/core/management/", line 351, in handle
return self.handle_noargs(**options)
File "/usr/lib/python2.4/site-packages/django/core/management/commands/", line 149, in handle_noargs
call_command('loaddata', 'initial_data', verbosity=verbosity)
File "/usr/lib/python2.4/site-packages/django/core/management/", line 158, in call_command
return klass.execute(*args, **options)
File "/usr/lib/python2.4/site-packages/django/core/management/", line 222, in execute
output = self.handle(*args, **options)
File "/usr/lib/python2.4/site-packages/django/core/management/commands/", line 91, in handle
formats = serializers.get_public_serializer_formats()
File "/usr/lib/python2.4/site-packages/django/core/serializers/", line 72, in get_public_serializer_formats
File "/usr/lib/python2.4/site-packages/django/core/serializers/", line 108, in _load_serializers
register_serializer(format, BUILTIN_SERIALIZERS[format], serializers)
File "/usr/lib/python2.4/site-packages/django/core/serializers/", line 50, in register_serializer
module = __import__(serializer_module, {}, {}, [''])
File "/usr/lib/python2.4/site-packages/django/core/serializers/", line 44, in ?
class DjangoJSONEncoder(simplejson.JSONEncoder):
AttributeError: 'module' object has no attribute 'JSONEncoder'

Googling says that the problem is that I have python-json installed, and that I need to have python-simplejson installed too. Except that I already do have that installed. So, when running a syncdb, just move in the Python site-packages directory aside for a moment, and everything should run fine.

sslerror: (8, 'EOF occurred in violation of protocol')

I have a program in the field that performs SOAP calls to a server over HTTPS. The program works fine at most places. However, for one customer, the relevant call gives an error of

sslerror: (8, 'EOF occurred in violation of protocol')

I started by taking a look at the stack trace on this guy:

Traceback (most recent call last):
File "/roadware/tire/custom/rt/dcpr", line 257, in main
products = q.getPrices(, customer_id=opts.customer,, pricing_type_id="1", products=opts.products)
File "/roadware/tire/custom/rt/dcpr", line 149, in getPrices
response =, gpelem, headerelem)
File "/usr/lib/python2.3/site-packages/elementsoap/", line 209, in call
File "/usr/lib/python2.3/site-packages/elementsoap/", line 147, in do_request
File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/", line 712, in endheaders
File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/", line 597, in _send_output
File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/", line 564, in send
File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/", line 985, in connect
ssl = socket.ssl(sock, self.key_file, self.cert_file)
File "/opt/K/SCO/python/2.3.4/usr/lib/python2.3/", line 73, in ssl
return _realssl(sock, keyfile, certfile)
sslerror: (8, 'EOF occurred in violation of protocol')

Googling around for this problem suggests that proxies can result in this problem, which is likely caused by the HTTP header send being terminated prematurely. A few people suggested using a different SSL library (M2Crypto), but my solution makes use of ElementSOAP, and I'd really rather not have to mess with that much of the underlying code. So I took a look at the source of these Python packages:


h.putrequest(method, path)
h.putheader("User-Agent", self.user_agent)
h.putheader("Content-Type", content_type)
h.putheader("Content-Length", str(len(body)))
if extra_headers:
for key, value in extra_headers:
h.putheader(key, value)


# fetch the reply
errcode, errmsg, headers = h.getreply()

It looks like the header send is blowing up before the body of the SOAP message goes--we're not going to be able to just suppress the exception and take our response from the server.

# call the server
response = self.__client.do_request(
extra_headers=[("SOAPAction", action)],
except HTTPError, v:
if v[0] == 500:
# might be a SOAP fault
response = namespace_parse(v[3])

Well, this is interesting. I'm not an expert on proxies or HTTP headers, but I see that SOAPAction extra header getting sent, and I can envision a proxy stripping out that header before sending it on to it's final destination. Result? Headers in violation of protocol (according to IIS)? Maybe. However, the customer claims there is no proxy.

My next thought? This post suggests changing the defaultHttpsTransport in from
httplib.HTTPConnection to M2Crypto.httpslib.HTTPSConnection. Unfortunately, I'm working on SCO Openserver, so I don't know if I'll be able to build M2Crypto for this to work.

Instead, I tried to investigate the problem further. On the non-functioning machine, I just tried to access the URL I am targeting using curl. The result?

curl: (1) libcurl was built with SSL disabled, https: not supported!

Really? Well, just be be sure this would work at all, I tried it on a machine that didn't have the problem.

curl: (60) error setting certificate verify locations:
CAfile: /usr/share/curl/curl-ca-bundle.crt
CApath: none

More details here:

curl performs SSL certificate verification by default, using a "bundle"
of Certificate Authority (CA) public keys (CA certs). The default
bundle is named curl-ca-bundle.crt; you can specify an alternate file
using the --cacert option.
If this HTTPS server uses a certificate signed by a CA represented in
the bundle, the certificate verification probably failed due to a
problem with the certificate (it might be expired, or the name might
not match the domain name in the URL).
If you'd like to turn off curl's verification of the certificate, use
the -k (or --insecure) option.

Coincidence? Or are we onto something? I find another machine that hasn't been running this program, and try the curl command.

curl: (1) libcurl was built with SSL disabled, https: not supported!

And the program gives sslerror: (8, 'EOF occurred in violation of protocol') on this system.

Maybe SSL isn't built the same on the failing machines?

One more time, but on a different machine that is working. This time, the curl command works.

It turns out the failing systems are all running SCO Openserver 5.0.6, while the working systems are running SCO Openserver 5.0.7. Maybe there's a problem with the SSL library installed on 5.0.6? Or the Python installation on those? Or something being used didn't have SSL compiled in, like curl? So maybe the python wrapper calling the SSL library isn't getting what it should be getting out of the library, and then is sending something wrong to the server instead of notifying that something went wrong earlier.

At this point, we've decided we are going to require 5.0.7 for this program, so I'm no longer actively researching a solution.

Code Syntax Highlighting on Blogger


If you want your code formatted in a Blogger post, just add the following two lines to your template, before the </body> tag.

<script src='' type='text/javascript'/>

<link href='' rel='stylesheet' type='text/css'/>

Then, just wrap any code in <pre> tags, make sure you add the name="code" attribute, and select the language you are formatting for in the class:

<pre name="code" class="brush: c-sharp;">
function test() : String
return 10;


function test() : String
return 10;


The first order of business for this blog was getting code syntax highlighting working. Not too difficult, there's a utility called SyntaxHighlighter that will do this in JavaScript, and instructions here and here on making it work in Blogger. Unfortunately, this still required hosting the Syntax Highlighter files somewhere, and adding some custom code from one of the above posts, or else Blogger adds visible <br/> tags to the line breaks in the code.

So, I combined all the relevant JavaScript from the Syntax Highlighter project (built against 2.0.287), added the necessary code to remove the <br/> tags, and put that up on a Google code project here. Then I just linked to that file (ebcsh.js) directly out of the svn trunk.

The css required a little more work, as Firefox refused to load css served from the Google Code's svn trunk. Google served the file with a MIME type of "text/plain", instead of "text/css", so I dialed up 1997 for some swanky free Geocities hosting.