The issue is that, when using the current libvncserver source, it is impossible to disable Tight JPEG encoding.
The way Tight/Turbo viewers disable JPEG encoding is by simply not sending the Tight quality value, causing the
server to use the default value of -1. Thus, cl->tightQualityLevel has to be set to -1 prior to processing the
encodings message for this mechanism to work. Similarly, it is not guaranteed that the compress level will be
set in the encodings message, so it is set to a default value prior to processing the message.
TurboVNC is a variant of TightVNC that uses the same client/server protocol (RFB version 3.8t),
and thus it is fully cross-compatible with TightVNC and TigerVNC (with one exception, which is noted below.)
Both the TightVNC and TurboVNC encoders analyze each rectangle, pick out regions of solid color to send
separately, and send the remaining subrectangles using mono, indexed color, JPEG, or raw encoding, depending
on the number of colors in the subrectangle. However, TurboVNC uses a fundamentally different selection
algorithm to determine the appropriate subencoding to use for each subrectangle. Thus, while it sends a
protocol stream that can be decoded by any TightVNC-compatible viewer, the mix of subencoding types in this
protocol stream will be different from those generated by a TightVNC server.
The research that led to TurboVNC is described in the following report:
http://www.virtualgl.org/pmwiki/uploads/About/tighttoturbo.pdf.
In summary: 20 RFB captures, representing "common" 2D and 3D application workloads (the 3D workloads were
run using VirtualGL), were studied using the TightVNC encoder in isolation. Some of the analysis features
in the TightVNC encoder, such as smoothness detection, were found to generate a lot of CPU usage with little
or no benefit in compression, so those features were disabled. JPEG encoding was accelerated using
libjpeg-turbo (which achieves a 2-4x speedup over plain libjpeg on modern x86 or ARM processors.) Finally,
the "palette threshold" (minimum number of colors that the subrectangle must have before it is compressed
using JPEG or raw) was adjusted to account for the fact that JPEG encoding is now quite a bit faster
(meaning that we can now use it more without a CPU penalty.) TurboVNC has additional optimizations,
such as the ability to count colors and encode JPEG images directly from the framebuffer without first
translating the pixels into RGB. The TurboVNC encoder compares quite favorably in terms of compression
ratio with TightVNC and generally encodes a great deal faster (often an order of magnitude or more.)
The version of the TurboVNC encoder included in this patch is roughly equivalent to the one found in version
0.6 of the Unix TurboVNC Server, with a few minor patches integrated from TurboVNC 1.1. TurboVNC 1.0
added multi-threading capabilities, which can be added in later if desired (at the expense of making
libvncserver depend on libpthread.)
Because TurboVNC uses a fundamentally different mix of subencodings than TightVNC, because it uses
the identical protocol (and thus a viewer really has no idea whether it's talking to a TightVNC or
TurboVNC server), and because it doesn't support rfbTightPng (and in fact conflicts with it-- see below),
the TurboVNC and TightVNC encoders cannot be enabled simultaneously.
Compatibility:
In *most* cases, a TurboVNC-enabled viewer is fully compatible with a TightVNC server, and vice versa.
TurboVNC supports pseudo-encodings for specifying a fine-grained (1-100) quality scale and specifying
chrominance subsampling. If a TurboVNC viewer sends those to a TightVNC server, then the TightVNC server
ignores them, so the TurboVNC viewer also sends the quality on a 0-9 scale that the TightVNC server can
understand. Similarly, the TurboVNC server checks first for fine-grained quality and subsampling
pseudo-encodings from the viewer, and failing to receive those, it then checks for the TightVNC 0-9
quality pseudo-encoding.
There is one case in which the two systems are not compatible, and that is when a TightVNC or TigerVNC
viewer requests compression level 0 without JPEG from a TurboVNC server. For performance reasons,
this causes the TurboVNC server to send images directly to the viewer, bypassing Zlib. When the
TurboVNC server does this, it also sets bits 7-4 in the compression control byte to rfbTightNoZlib (0x0A),
which is unfortunately the same value as rfbTightPng. Older TightVNC viewers that don't handle PNG
will assume that the stream is uncompressed but still encapsulated in a Zlib structure, whereas newer
PNG-supporting TightVNC viewers will assume that the stream is PNG. In either case, the viewer will
probably crash. Since most VNC viewers don't expose compression level 0 in the GUI, this is a
relatively rare situation.
Description of changes:
configure.ac
-- Added support for libjpeg-turbo. If passed an argument of --with-turbovnc, configure will now run
(or, if cross-compiling, just link) a test program that determines whether the libjpeg library being
used is libjpeg-turbo. libjpeg-turbo must be used when building the TurboVNC encoder, because the
TurboVNC encoder relies on the libjpeg-turbo colorspace extensions in order to compress images directly
out of the framebuffer (which may be, for instance, BGRA rather than RGB.) libjpeg-turbo can optionally
be used with the TightVNC encoder as well, but the speedup will only be marginal (the report linked
above explains why in more detail, but basically it's because of Amdahl's Law. The TightVNC encoder
was designed with the assumption that JPEG had a very high CPU cost, and thus JPEG is used only sparingly.)
-- Added a new configure variable, JPEG_LDFLAGS. This is necessitated by the fact that libjpeg-turbo
often distributes libjpeg.a and libjpeg.so in /opt/libjpeg-turbo/lib32 or /opt/libjpeg-turbo/lib64,
and many people prefer to statically link with it. Thus, more flexibility is needed than is provided
by --with-jpeg. If JPEG_LDFLAGS is specified, then it overrides the changes to LDFLAGS enacted by
--with-jpeg (but --with-jpeg is still used to set the include path.) The addition of JPEG_LDFLAGS
necessitated replacing AC_CHECK_LIB with AC_LINK_IFELSE (because AC_CHECK_LIB automatically sets
LIBS to -ljpeg, which is not what we want if we're, for instance, linking statically with libjpeg-turbo.)
-- configure does not check for PNG support if TurboVNC encoding is enabled. This prevents the
rfbSendRectEncodingTightPng() function from being compiled in, since the TurboVNC encoder doesn't
(and can't) support it.
common/turbojpeg.c, common/turbojpeg.h
-- TurboJPEG is a simple API used to compress and decompress JPEG images in memory. It was originally
implemented because it was desirable to use different types of underlying technologies to compress
JPEG on different platforms (mediaLib on SPARC, Quicktime on PPC Macs, Intel Performance Primitives, etc.)
These days, however, libjpeg-turbo is the only underlying technology used by TurboVNC, so TurboJPEG's
purpose is largely just code simplicity and flexibility. Thus, since there is no real need for
libvncserver to use any technology other than libjpeg-turbo for compressing JPEG, the TurboJPEG wrapper
for libjpeg-turbo has been included in-tree so that libvncserver can be directly linked with libjpeg-turbo.
This is convenient because many modern Linux distros (Fedora, Ubuntu, etc.) now ship libjpeg-turbo as
their default libjpeg library.
libvncserver/rfbserver.c
-- Added logic to check for the TurboVNC fine-grained quality level and subsampling encodings and to
map Tight (0-9) quality levels to appropriate fine-grained quality level and subsampling values if
communicating with a TightVNC/TigerVNC viewer.
libvncserver/turbo.c
-- TurboVNC encoder (compiled instead of libvncserver/tight.c)
rfb/rfb.h
-- Added support for the TurboVNC subsampling level
rfb/rfbproto.h
-- Added constants for the TurboVNC fine quality level and subsampling encodings as well as the rfbTightNoZlib
constant and notes on its usage.
Support connections from the Mac OS X built-in VNC client to
LibVNCServers running with no password and advertising a server
version of 3.7 or greater.
The png stuff in tight.c depends on code in tight.c that uses libjpeg
features. We could probably seperate that, but for now the dependency
for 'tight' goes:
PNG depends on JPEG depends on ZLIB.
This is reflected in Makefile.am now.
NB: Building tight.c with JPEG but without PNG is still possible,
but nor the other way around.
Lengthy explanation follows...
First, the scenario before this patch:
We have three clients 1,2,3 connected. The main thread loops through
them using rfbClientIteratorNext() (loop L1) and is currently at
client 2 i.e. client 2's cl_2->refCount is 1. At this point we need to
loop again through the clients, with cl_2->refCount == 1, i.e. do a
loop L2 nested within loop L1.
BUT: Now client 2 disconnects, it's clientInput thread terminates its
clientOutput thread and calls rfbClientConnectionGone(). This LOCKs
clientListMutex and WAITs for cl_2->refCount to become 0. This means
this thread waits for the main thread to release cl_2. Waiting, with
clientListMutex LOCKed!
Meanwhile, the main thread is about to begin the inner
rfbClientIteratorNext() loop L2. The first call to rfbClientIteratorNext()
LOCKs clientListMutex. BAAM. This mutex is locked by cl2's clientInput
thread and is only released when cl_2->refCount becomes 0. The main thread
would decrement cl_2->refCount when it would continue with loop L1. But
it's waiting for cl2's clientInput thread to release clientListMutex. Which
never happens since this one's waiting for the main thread to decrement
cl_2->refCount. DEADLOCK.
Now, situation with this patch:
Same as above, but when client 2 disconnects it's clientInput thread
rfbClientConnectionGone(). This again LOCKs clientListMutex, removes cl_2
from the linked list and UNLOCKS clientListMutex. The WAIT for
cl_2->refCount to become 0 is _after_ that. Waiting, with
clientListMutex UNLOCKed!
Therefore, the main thread can continue, do the inner loop L2 (now only
looping through 1,3 - 2 was removed from the linked list) and continue with
loop L1, finally decrementing cl_2->refCount, allowing cl2's clientInput
thread to continue and terminate. The resources held by cl2 are not free()'d
by rfbClientConnectionGone until cl2->refCount becomes 0, i.e. loop L1 has
released cl2.
I was debbuging some code tonight and i found a pointer that is not been
freed, so i think there is maybe a memory leak, so it is...
there is the malloc caller reverse order:
( malloc cl->statEncList )
<- rfbStatLookupEncoding
<- rfbStatRecordEncodingSent
<- rfbSendCursorPos
<- rfbSendFramebufferUpdate
<- rfbProcessEvents
I didnt look the whole libvncserver api, but i am using
rfbReverseConnection with rfbProcessEvents, and then when the client
connection dies, i am calling a rfbShutdownServer and rfbScreenCleanup,
but the malloc at rfbStatLookupEncoding isnt been freed.
So to free the stats i added a rfbResetStats(cl) after rfbPrintStats(cl)
at rfbClientConnectionGone in rfbserver.c before free the cl pointer. (at
rfbserver.c line 555). And this, obviously, is correcting the memory leak.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
We do this simply by omitting the content-type and let the browser
decide upon the mime-type of the sent file. Only exception is
'index.vnc', where we do set the content-type since some browsers
fail to detect it's html when it's ending in '.vnc'
Also, remove superfluous #defines. We close the connection always.
Add common/sha1.h and common/sha1.c so that we have the SHA routines
even if openssl is not available. From the IETF SHA RFC example code.
Remove the UTF-8 encoding hack. This was really just an experiment.
If the protocol passed in the handshake has "binary" then don't base64
encode for the HyBi protocol. This will allow noVNC to request the
binary data be passed raw and not base64 encoded. Unfortunately, the
client doesn't speak first in VNC protocol (bad original design). If
it did then we could determine whether to base64 encode or not based
on the first HyBi frame from the client and whether the binary bit is
set or not. Oh well.
Misc Cleanup:
- Always free response and buf in handshake routine.
- Remove some unused variables.
Move the hixie disconnect hack to websockets.c. Removed
the remaining websockets vars from rfbClientPtr, so all
websockets stuff is hidden behind an opaque pointer.
Added wspath member to rfbClientRec which holds the
path component of the initial websocket request.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
- Add --sslcertfile and --sslkeyfile. These should really be combined
with the existing x11vnc command line options for SSL support.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
For now, only OpenSSL support is activated through configure, since GnuTLS
is only used in LibVNCClient.
[jes: separated this out from the commit adding encryption support, added
autoconf support.]
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
[jes: moved out GnuTLS and OpenSSL support, added a dummy support, to
separate changes better, and to keep things compiling]
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is not completely standard UTF-8 encoding. Only code points 0-255
are encoded and never encoded to more than two octets. Since '\x00' is
a WebSockets framing character, it's easier for all parties to encode
zero as '\xc4\x80', i.e. 194+128, i.e. UTF-8 256.
This means that a random stream will be slightly more than 50% larger
using this encoding scheme. But it's easy CPU-wise for client and
server to decode/encode. This is especially important for clients
written in languages that have weak bitops, like Javascript (i.e. the
noVNC client).
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
If the only thing we are waiting on is a WebSockets terminator, then
remove it from the stream early on in rfbProcessClientNormalMessage.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Has a bug: WebSocket client disconnects are not detected.
rfbSendFramebufferUpdate is doing a MSG_PEEK recv to determine if
enough data is available which prevents a disconnect from being
detected.
Otherwise it's working pretty well.
[jes: moved added struct members to the end for binary compatibility with
previous LibVNCServer versions, removed an unused variable]
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>
This is required to be able to do proper event loop integration with Qt.
Idea was taken from vino's libvncserver fork.
Signed-off-by: Christian Beier <dontmind@freeshell.org>
This bug occured when a second telepathy tubes client was connected after
the first one had disconnected and the channel (thus, the screen too)
had been destroyed.
Signed-off-by: Johannes Schindelin <johannes.schindelin@gmx.de>