While browsing Linux kernel source code I came across POSIX (Portable Operating System Interface) variable called _POSIX_HOST_NAME_MAX . It defines the maximum length of a host name (not including the terminating null) as returned from the gethostname function.
This variable sets the limit to the number of characters in a hostname under Linux.
$ grep '_POSIX_HOST_NAME_MAX' /usr/include/bits/posix1_lim.h
#define _POSIX_HOST_NAME_MAX 255
You can change limit and recompile the kernel to get bigger hostname.
See comment below for correct information.
Updated for accuracy.