You can not select more than 25 topics Topics must start with a letter or number, can include dashes ('-') and can be up to 35 characters long.
libtdevnc/rfb/rfbproto.h

1463 lines
51 KiB

#ifndef RFBPROTO_H
#define RFBPROTO_H
/**
@mainpage
@li @ref libvncserver_api
@li @ref libvncserver_doc
@li @ref libvncclient_api
@li @ref libvncclient_doc
*/
22 years ago
/*
Add TurboVNC encoding support. TurboVNC is a variant of TightVNC that uses the same client/server protocol (RFB version 3.8t), and thus it is fully cross-compatible with TightVNC and TigerVNC (with one exception, which is noted below.) Both the TightVNC and TurboVNC encoders analyze each rectangle, pick out regions of solid color to send separately, and send the remaining subrectangles using mono, indexed color, JPEG, or raw encoding, depending on the number of colors in the subrectangle. However, TurboVNC uses a fundamentally different selection algorithm to determine the appropriate subencoding to use for each subrectangle. Thus, while it sends a protocol stream that can be decoded by any TightVNC-compatible viewer, the mix of subencoding types in this protocol stream will be different from those generated by a TightVNC server. The research that led to TurboVNC is described in the following report: http://www.virtualgl.org/pmwiki/uploads/About/tighttoturbo.pdf. In summary: 20 RFB captures, representing "common" 2D and 3D application workloads (the 3D workloads were run using VirtualGL), were studied using the TightVNC encoder in isolation. Some of the analysis features in the TightVNC encoder, such as smoothness detection, were found to generate a lot of CPU usage with little or no benefit in compression, so those features were disabled. JPEG encoding was accelerated using libjpeg-turbo (which achieves a 2-4x speedup over plain libjpeg on modern x86 or ARM processors.) Finally, the "palette threshold" (minimum number of colors that the subrectangle must have before it is compressed using JPEG or raw) was adjusted to account for the fact that JPEG encoding is now quite a bit faster (meaning that we can now use it more without a CPU penalty.) TurboVNC has additional optimizations, such as the ability to count colors and encode JPEG images directly from the framebuffer without first translating the pixels into RGB. The TurboVNC encoder compares quite favorably in terms of compression ratio with TightVNC and generally encodes a great deal faster (often an order of magnitude or more.) The version of the TurboVNC encoder included in this patch is roughly equivalent to the one found in version 0.6 of the Unix TurboVNC Server, with a few minor patches integrated from TurboVNC 1.1. TurboVNC 1.0 added multi-threading capabilities, which can be added in later if desired (at the expense of making libvncserver depend on libpthread.) Because TurboVNC uses a fundamentally different mix of subencodings than TightVNC, because it uses the identical protocol (and thus a viewer really has no idea whether it's talking to a TightVNC or TurboVNC server), and because it doesn't support rfbTightPng (and in fact conflicts with it-- see below), the TurboVNC and TightVNC encoders cannot be enabled simultaneously. Compatibility: In *most* cases, a TurboVNC-enabled viewer is fully compatible with a TightVNC server, and vice versa. TurboVNC supports pseudo-encodings for specifying a fine-grained (1-100) quality scale and specifying chrominance subsampling. If a TurboVNC viewer sends those to a TightVNC server, then the TightVNC server ignores them, so the TurboVNC viewer also sends the quality on a 0-9 scale that the TightVNC server can understand. Similarly, the TurboVNC server checks first for fine-grained quality and subsampling pseudo-encodings from the viewer, and failing to receive those, it then checks for the TightVNC 0-9 quality pseudo-encoding. There is one case in which the two systems are not compatible, and that is when a TightVNC or TigerVNC viewer requests compression level 0 without JPEG from a TurboVNC server. For performance reasons, this causes the TurboVNC server to send images directly to the viewer, bypassing Zlib. When the TurboVNC server does this, it also sets bits 7-4 in the compression control byte to rfbTightNoZlib (0x0A), which is unfortunately the same value as rfbTightPng. Older TightVNC viewers that don't handle PNG will assume that the stream is uncompressed but still encapsulated in a Zlib structure, whereas newer PNG-supporting TightVNC viewers will assume that the stream is PNG. In either case, the viewer will probably crash. Since most VNC viewers don't expose compression level 0 in the GUI, this is a relatively rare situation. Description of changes: configure.ac -- Added support for libjpeg-turbo. If passed an argument of --with-turbovnc, configure will now run (or, if cross-compiling, just link) a test program that determines whether the libjpeg library being used is libjpeg-turbo. libjpeg-turbo must be used when building the TurboVNC encoder, because the TurboVNC encoder relies on the libjpeg-turbo colorspace extensions in order to compress images directly out of the framebuffer (which may be, for instance, BGRA rather than RGB.) libjpeg-turbo can optionally be used with the TightVNC encoder as well, but the speedup will only be marginal (the report linked above explains why in more detail, but basically it's because of Amdahl's Law. The TightVNC encoder was designed with the assumption that JPEG had a very high CPU cost, and thus JPEG is used only sparingly.) -- Added a new configure variable, JPEG_LDFLAGS. This is necessitated by the fact that libjpeg-turbo often distributes libjpeg.a and libjpeg.so in /opt/libjpeg-turbo/lib32 or /opt/libjpeg-turbo/lib64, and many people prefer to statically link with it. Thus, more flexibility is needed than is provided by --with-jpeg. If JPEG_LDFLAGS is specified, then it overrides the changes to LDFLAGS enacted by --with-jpeg (but --with-jpeg is still used to set the include path.) The addition of JPEG_LDFLAGS necessitated replacing AC_CHECK_LIB with AC_LINK_IFELSE (because AC_CHECK_LIB automatically sets LIBS to -ljpeg, which is not what we want if we're, for instance, linking statically with libjpeg-turbo.) -- configure does not check for PNG support if TurboVNC encoding is enabled. This prevents the rfbSendRectEncodingTightPng() function from being compiled in, since the TurboVNC encoder doesn't (and can't) support it. common/turbojpeg.c, common/turbojpeg.h -- TurboJPEG is a simple API used to compress and decompress JPEG images in memory. It was originally implemented because it was desirable to use different types of underlying technologies to compress JPEG on different platforms (mediaLib on SPARC, Quicktime on PPC Macs, Intel Performance Primitives, etc.) These days, however, libjpeg-turbo is the only underlying technology used by TurboVNC, so TurboJPEG's purpose is largely just code simplicity and flexibility. Thus, since there is no real need for libvncserver to use any technology other than libjpeg-turbo for compressing JPEG, the TurboJPEG wrapper for libjpeg-turbo has been included in-tree so that libvncserver can be directly linked with libjpeg-turbo. This is convenient because many modern Linux distros (Fedora, Ubuntu, etc.) now ship libjpeg-turbo as their default libjpeg library. libvncserver/rfbserver.c -- Added logic to check for the TurboVNC fine-grained quality level and subsampling encodings and to map Tight (0-9) quality levels to appropriate fine-grained quality level and subsampling values if communicating with a TightVNC/TigerVNC viewer. libvncserver/turbo.c -- TurboVNC encoder (compiled instead of libvncserver/tight.c) rfb/rfb.h -- Added support for the TurboVNC subsampling level rfb/rfbproto.h -- Added constants for the TurboVNC fine quality level and subsampling encodings as well as the rfbTightNoZlib constant and notes on its usage.
11 years ago
* Copyright (C) 2009-2010 D. R. Commander. All Rights Reserved.
* Copyright (C) 2005 Rohit Kumar, Johannes E. Schindelin
Add TurboVNC encoding support. TurboVNC is a variant of TightVNC that uses the same client/server protocol (RFB version 3.8t), and thus it is fully cross-compatible with TightVNC and TigerVNC (with one exception, which is noted below.) Both the TightVNC and TurboVNC encoders analyze each rectangle, pick out regions of solid color to send separately, and send the remaining subrectangles using mono, indexed color, JPEG, or raw encoding, depending on the number of colors in the subrectangle. However, TurboVNC uses a fundamentally different selection algorithm to determine the appropriate subencoding to use for each subrectangle. Thus, while it sends a protocol stream that can be decoded by any TightVNC-compatible viewer, the mix of subencoding types in this protocol stream will be different from those generated by a TightVNC server. The research that led to TurboVNC is described in the following report: http://www.virtualgl.org/pmwiki/uploads/About/tighttoturbo.pdf. In summary: 20 RFB captures, representing "common" 2D and 3D application workloads (the 3D workloads were run using VirtualGL), were studied using the TightVNC encoder in isolation. Some of the analysis features in the TightVNC encoder, such as smoothness detection, were found to generate a lot of CPU usage with little or no benefit in compression, so those features were disabled. JPEG encoding was accelerated using libjpeg-turbo (which achieves a 2-4x speedup over plain libjpeg on modern x86 or ARM processors.) Finally, the "palette threshold" (minimum number of colors that the subrectangle must have before it is compressed using JPEG or raw) was adjusted to account for the fact that JPEG encoding is now quite a bit faster (meaning that we can now use it more without a CPU penalty.) TurboVNC has additional optimizations, such as the ability to count colors and encode JPEG images directly from the framebuffer without first translating the pixels into RGB. The TurboVNC encoder compares quite favorably in terms of compression ratio with TightVNC and generally encodes a great deal faster (often an order of magnitude or more.) The version of the TurboVNC encoder included in this patch is roughly equivalent to the one found in version 0.6 of the Unix TurboVNC Server, with a few minor patches integrated from TurboVNC 1.1. TurboVNC 1.0 added multi-threading capabilities, which can be added in later if desired (at the expense of making libvncserver depend on libpthread.) Because TurboVNC uses a fundamentally different mix of subencodings than TightVNC, because it uses the identical protocol (and thus a viewer really has no idea whether it's talking to a TightVNC or TurboVNC server), and because it doesn't support rfbTightPng (and in fact conflicts with it-- see below), the TurboVNC and TightVNC encoders cannot be enabled simultaneously. Compatibility: In *most* cases, a TurboVNC-enabled viewer is fully compatible with a TightVNC server, and vice versa. TurboVNC supports pseudo-encodings for specifying a fine-grained (1-100) quality scale and specifying chrominance subsampling. If a TurboVNC viewer sends those to a TightVNC server, then the TightVNC server ignores them, so the TurboVNC viewer also sends the quality on a 0-9 scale that the TightVNC server can understand. Similarly, the TurboVNC server checks first for fine-grained quality and subsampling pseudo-encodings from the viewer, and failing to receive those, it then checks for the TightVNC 0-9 quality pseudo-encoding. There is one case in which the two systems are not compatible, and that is when a TightVNC or TigerVNC viewer requests compression level 0 without JPEG from a TurboVNC server. For performance reasons, this causes the TurboVNC server to send images directly to the viewer, bypassing Zlib. When the TurboVNC server does this, it also sets bits 7-4 in the compression control byte to rfbTightNoZlib (0x0A), which is unfortunately the same value as rfbTightPng. Older TightVNC viewers that don't handle PNG will assume that the stream is uncompressed but still encapsulated in a Zlib structure, whereas newer PNG-supporting TightVNC viewers will assume that the stream is PNG. In either case, the viewer will probably crash. Since most VNC viewers don't expose compression level 0 in the GUI, this is a relatively rare situation. Description of changes: configure.ac -- Added support for libjpeg-turbo. If passed an argument of --with-turbovnc, configure will now run (or, if cross-compiling, just link) a test program that determines whether the libjpeg library being used is libjpeg-turbo. libjpeg-turbo must be used when building the TurboVNC encoder, because the TurboVNC encoder relies on the libjpeg-turbo colorspace extensions in order to compress images directly out of the framebuffer (which may be, for instance, BGRA rather than RGB.) libjpeg-turbo can optionally be used with the TightVNC encoder as well, but the speedup will only be marginal (the report linked above explains why in more detail, but basically it's because of Amdahl's Law. The TightVNC encoder was designed with the assumption that JPEG had a very high CPU cost, and thus JPEG is used only sparingly.) -- Added a new configure variable, JPEG_LDFLAGS. This is necessitated by the fact that libjpeg-turbo often distributes libjpeg.a and libjpeg.so in /opt/libjpeg-turbo/lib32 or /opt/libjpeg-turbo/lib64, and many people prefer to statically link with it. Thus, more flexibility is needed than is provided by --with-jpeg. If JPEG_LDFLAGS is specified, then it overrides the changes to LDFLAGS enacted by --with-jpeg (but --with-jpeg is still used to set the include path.) The addition of JPEG_LDFLAGS necessitated replacing AC_CHECK_LIB with AC_LINK_IFELSE (because AC_CHECK_LIB automatically sets LIBS to -ljpeg, which is not what we want if we're, for instance, linking statically with libjpeg-turbo.) -- configure does not check for PNG support if TurboVNC encoding is enabled. This prevents the rfbSendRectEncodingTightPng() function from being compiled in, since the TurboVNC encoder doesn't (and can't) support it. common/turbojpeg.c, common/turbojpeg.h -- TurboJPEG is a simple API used to compress and decompress JPEG images in memory. It was originally implemented because it was desirable to use different types of underlying technologies to compress JPEG on different platforms (mediaLib on SPARC, Quicktime on PPC Macs, Intel Performance Primitives, etc.) These days, however, libjpeg-turbo is the only underlying technology used by TurboVNC, so TurboJPEG's purpose is largely just code simplicity and flexibility. Thus, since there is no real need for libvncserver to use any technology other than libjpeg-turbo for compressing JPEG, the TurboJPEG wrapper for libjpeg-turbo has been included in-tree so that libvncserver can be directly linked with libjpeg-turbo. This is convenient because many modern Linux distros (Fedora, Ubuntu, etc.) now ship libjpeg-turbo as their default libjpeg library. libvncserver/rfbserver.c -- Added logic to check for the TurboVNC fine-grained quality level and subsampling encodings and to map Tight (0-9) quality levels to appropriate fine-grained quality level and subsampling values if communicating with a TightVNC/TigerVNC viewer. libvncserver/turbo.c -- TurboVNC encoder (compiled instead of libvncserver/tight.c) rfb/rfb.h -- Added support for the TurboVNC subsampling level rfb/rfbproto.h -- Added constants for the TurboVNC fine quality level and subsampling encodings as well as the rfbTightNoZlib constant and notes on its usage.
11 years ago
* Copyright (C) 2004-2008 Sun Microsystems, Inc. All Rights Reserved.
* Copyright (C) 2000-2002 Constantin Kaplinsky. All Rights Reserved.
22 years ago
* Copyright (C) 2000 Tridia Corporation. All Rights Reserved.
* Copyright (C) 1999 AT&T Laboratories Cambridge. All Rights Reserved.
*
* This is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* This software is distributed in the hope that it will be useful,
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this software; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place - Suite 330, Boston, MA 02111-1307,
* USA.
*/
/*
* rfbproto.h - header file for the RFB protocol version 3.3
*
* Uses types CARD<n> for an n-bit unsigned integer, INT<n> for an n-bit signed
* integer (for n = 8, 16 and 32).
*
* All multiple byte integers are in big endian (network) order (most
* significant byte first). Unless noted otherwise there is no special
* alignment of protocol structures.
*
*
* Once the initial handshaking is done, all messages start with a type byte,
* (usually) followed by message-specific data. The order of definitions in
* this file is as follows:
*
* (1) Structures used in several types of message.
* (2) Structures used in the initial handshaking.
* (3) Message types.
* (4) Encoding types.
* (5) For each message type, the form of the data following the type byte.
* Sometimes this is defined by a single structure but the more complex
* messages have to be explained by comments.
*/
#include <stdint.h>
22 years ago
18 years ago
#if defined(WIN32) && !defined(__MINGW32__)
#define LIBVNCSERVER_WORDS_BIGENDIAN
#define rfbBool int
#include <sys/timeb.h>
#include <winsock2.h>
#undef SOCKET
#define SOCKET int
#else
#include <rfb/rfbconfig.h>
#endif
#ifdef LIBVNCSERVER_HAVE_LIBZ
#include <zlib.h>
#ifdef __CHECKER__
#undef Z_NULL
#define Z_NULL NULL
#endif
#endif
#if !defined(_WIN32)
# include <endian.h>
# if __BYTE_ORDER == __BIG_ENDIAN
# define LIBVNCSERVER_WORDS_BIGENDIAN 1
# endif
#endif /* !_WIN32 */
/* MS compilers don't have strncasecmp */
#ifdef _MSC_VER
#define strncasecmp _strnicmp
#endif
#define rfbMax(a,b) (((a)>(b))?(a):(b))
18 years ago
#if !defined(WIN32) || defined(__MINGW32__)
#ifdef LIBVNCSERVER_HAVE_SYS_TIME_H
#include <sys/time.h>
#endif
#ifdef LIBVNCSERVER_HAVE_NETINET_IN_H
#include <netinet/in.h>
#endif
#define SOCKET int
typedef int8_t rfbBool;
#undef FALSE
#define FALSE 0
#undef TRUE
#define TRUE -1
#endif
typedef uint32_t rfbKeySym;
typedef uint32_t rfbPixel;
#ifdef LIBVNCSERVER_NEED_INADDR_T
typedef uint32_t in_addr_t;
#endif
#ifndef INADDR_NONE
#define INADDR_NONE ((in_addr_t) 0xffffffff)
#endif
#define MAX_ENCODINGS 64
22 years ago
/*****************************************************************************
*
* Structures used in several messages
*
*****************************************************************************/
/*-----------------------------------------------------------------------------
* Structure used to specify a rectangle. This structure is a multiple of 4
* bytes so that it can be interspersed with 32-bit pixel data without
* affecting alignment.
*/
typedef struct {
uint16_t x;
uint16_t y;
uint16_t w;
uint16_t h;
22 years ago
} rfbRectangle;
#define sz_rfbRectangle 8
/*-----------------------------------------------------------------------------
* Structure used to specify pixel format.
*/
typedef struct {
uint8_t bitsPerPixel; /* 8,16,32 only */
22 years ago
uint8_t depth; /* 8 to 32 */
22 years ago
uint8_t bigEndian; /* True if multi-byte pixels are interpreted
22 years ago
as big endian, or if single-bit-per-pixel
has most significant bit of the byte
corresponding to first (leftmost) pixel. Of
course this is meaningless for 8 bits/pix */
uint8_t trueColour; /* If false then we need a "colour map" to
22 years ago
convert pixels to RGB. If true, xxxMax and
xxxShift specify bits used for red, green
and blue */
/* the following fields are only meaningful if trueColour is true */
uint16_t redMax; /* maximum red value (= 2^n - 1 where n is the
22 years ago
number of bits used for red). Note this
value is always in big endian order. */
uint16_t greenMax; /* similar for green */
22 years ago
uint16_t blueMax; /* and blue */
22 years ago
uint8_t redShift; /* number of shifts needed to get the red
22 years ago
value in a pixel to the least significant
bit. To find the red value from a given
pixel, do the following:
1) Swap pixel value according to bigEndian
(e.g. if bigEndian is false and host byte
order is big endian, then swap).
2) Shift right by redShift.
3) AND with redMax (in host byte order).
4) You now have the red value between 0 and
redMax. */
uint8_t greenShift; /* similar for green */
22 years ago
uint8_t blueShift; /* and blue */
22 years ago
uint8_t pad1;
uint16_t pad2;
22 years ago
} rfbPixelFormat;
#define sz_rfbPixelFormat 16
/* UltraVNC: Color settings values */
#define rfbPFFullColors 0
#define rfbPF256Colors 1
#define rfbPF64Colors 2
#define rfbPF8Colors 3
#define rfbPF8GreyColors 4
#define rfbPF4GreyColors 5
#define rfbPF2GreyColors 6
22 years ago
/*****************************************************************************
*
* Initial handshaking messages
*
*****************************************************************************/
/*-----------------------------------------------------------------------------
* Protocol Version
*
* The server always sends 12 bytes to start which identifies the latest RFB
* protocol version number which it supports. These bytes are interpreted
* as a string of 12 ASCII characters in the format "RFB xxx.yyy\n" where
* xxx and yyy are the major and minor version numbers (for version 3.3
* this is "RFB 003.003\n").
*
* The client then replies with a similar 12-byte message giving the version
* number of the protocol which should actually be used (which may be different
* to that quoted by the server).
*
* It is intended that both clients and servers may provide some level of
* backwards compatibility by this mechanism. Servers in particular should
* attempt to provide backwards compatibility, and even forwards compatibility
* to some extent. For example if a client demands version 3.1 of the
* protocol, a 3.0 server can probably assume that by ignoring requests for
* encoding types it doesn't understand, everything will still work OK. This
* will probably not be the case for changes in the major version number.
*
* The format string below can be used in sprintf or sscanf to generate or
* decode the version string respectively.
*/
#define rfbProtocolVersionFormat "RFB %03d.%03d\n"
#define rfbProtocolMajorVersion 3
#define rfbProtocolMinorVersion 8
/* UltraVNC Viewer examines rfbProtocolMinorVersion number (4, and 6)
* to identify if the server supports File Transfer
*/
22 years ago
typedef char rfbProtocolVersionMsg[13]; /* allow extra byte for null */
#define sz_rfbProtocolVersionMsg 12
/*
* Negotiation of the security type (protocol version 3.7)
*
* Once the protocol version has been decided, the server either sends a list
* of supported security types, or informs the client about an error (when the
* number of security types is 0). Security type rfbSecTypeTight is used to
* enable TightVNC-specific protocol extensions. The value rfbSecTypeVncAuth
* stands for classic VNC authentication.
*
* The client selects a particular security type from the list provided by the
* server.
*/
#define rfbSecTypeInvalid 0
#define rfbSecTypeNone 1
#define rfbSecTypeVncAuth 2
22 years ago
/*-----------------------------------------------------------------------------
* Authentication
*
* Once the protocol version has been decided, the server then sends a 32-bit
* word indicating whether any authentication is needed on the connection.
* The value of this word determines the authentication scheme in use. For
* version 3.0 of the protocol this may have one of the following values:
*/
#define rfbConnFailed 0
#define rfbNoAuth 1
#define rfbVncAuth 2
#define rfbRA2 5
#define rfbRA2ne 6
#define rfbSSPI 7
#define rfbSSPIne 8
#define rfbTight 16
#define rfbUltra 17
#define rfbTLS 18
#define rfbVeNCrypt 19
#define rfbARD 30
#define rfbMSLogon 0xfffffffa
#define rfbVeNCryptPlain 256
#define rfbVeNCryptTLSNone 257
#define rfbVeNCryptTLSVNC 258
#define rfbVeNCryptTLSPlain 259
#define rfbVeNCryptX509None 260
#define rfbVeNCryptX509VNC 261
#define rfbVeNCryptX509Plain 262
#define rfbVeNCryptX509SASL 263
#define rfbVeNCryptTLSSASL 264
22 years ago
/*
* rfbConnFailed: For some reason the connection failed (e.g. the server
* cannot support the desired protocol version). This is
* followed by a string describing the reason (where a
* string is specified as a 32-bit length followed by that
* many ASCII characters).
*
* rfbNoAuth: No authentication is needed.
*
* rfbVncAuth: The VNC authentication scheme is to be used. A 16-byte
* challenge follows, which the client encrypts as
* appropriate using the password and sends the resulting
* 16-byte response. If the response is correct, the
* server sends the 32-bit word rfbVncAuthOK. If a simple
* failure happens, the server sends rfbVncAuthFailed and
* closes the connection. If the server decides that too
* many failures have occurred, it sends rfbVncAuthTooMany
* and closes the connection. In the latter case, the
* server should not allow an immediate reconnection by
* the client.
*/
#define rfbVncAuthOK 0
#define rfbVncAuthFailed 1
#define rfbVncAuthTooMany 2
/*-----------------------------------------------------------------------------
* Client Initialisation Message
*
* Once the client and server are sure that they're happy to talk to one
* another, the client sends an initialisation message. At present this
* message only consists of a boolean indicating whether the server should try
* to share the desktop by leaving other clients connected, or give exclusive
* access to this client by disconnecting all other clients.
*/
typedef struct {
uint8_t shared;
22 years ago
} rfbClientInitMsg;
#define sz_rfbClientInitMsg 1
/*-----------------------------------------------------------------------------
* Server Initialisation Message
*
* After the client initialisation message, the server sends one of its own.
* This tells the client the width and height of the server's framebuffer,
* its pixel format and the name associated with the desktop.
*/
typedef struct {
uint16_t framebufferWidth;
uint16_t framebufferHeight;
22 years ago
rfbPixelFormat format; /* the server's preferred pixel format */
uint32_t nameLength;
22 years ago
/* followed by char name[nameLength] */
} rfbServerInitMsg;
#define sz_rfbServerInitMsg (8 + sz_rfbPixelFormat)
/*
* Following the server initialisation message it's up to the client to send
* whichever protocol messages it wants. Typically it will send a
* SetPixelFormat message and a SetEncodings message, followed by a
* FramebufferUpdateRequest. From then on the server will send
* FramebufferUpdate messages in response to the client's
* FramebufferUpdateRequest messages. The client should send
* FramebufferUpdateRequest messages with incremental set to true when it has
* finished processing one FramebufferUpdate and is ready to process another.
* With a fast client, the rate at which FramebufferUpdateRequests are sent
* should be regulated to avoid hogging the network.
*/
/*****************************************************************************
*
* Message types
*
*****************************************************************************/
/* server -> client */
#define rfbFramebufferUpdate 0
#define rfbSetColourMapEntries 1
#define rfbBell 2
#define rfbServerCutText 3
/* Modif sf@2002 */
#define rfbResizeFrameBuffer 4
#define rfbPalmVNCReSizeFrameBuffer 0xF
22 years ago
/* client -> server */
#define rfbSetPixelFormat 0
#define rfbFixColourMapEntries 1 /* not currently supported */
#define rfbSetEncodings 2
#define rfbFramebufferUpdateRequest 3
#define rfbKeyEvent 4
#define rfbPointerEvent 5
#define rfbClientCutText 6
/* Modif sf@2002 - actually bidirectionnal */
#define rfbFileTransfer 7
/* Modif sf@2002 */
#define rfbSetScale 8
/* Modif rdv@2002 */
#define rfbSetServerInput 9
/* Modif rdv@2002 */
#define rfbSetSW 10
/* Modif sf@2002 - TextChat - Bidirectionnal */
#define rfbTextChat 11
/* Modif cs@2005 */
/* PalmVNC 1.4 & 2.0 SetScale Factor message */
#define rfbPalmVNCSetScaleFactor 0xF
/* Xvp message - bidirectional */
#define rfbXvp 250
22 years ago
/*****************************************************************************
*
* Encoding types
*
*****************************************************************************/
#define rfbEncodingRaw 0
#define rfbEncodingCopyRect 1
#define rfbEncodingRRE 2
#define rfbEncodingCoRRE 4
#define rfbEncodingHextile 5
#define rfbEncodingZlib 6
#define rfbEncodingTight 7
#define rfbEncodingTightPng 0xFFFFFEFC /* -260 */
22 years ago
#define rfbEncodingZlibHex 8
#define rfbEncodingUltra 9
#define rfbEncodingZRLE 16
#define rfbEncodingZYWRLE 17
22 years ago
#define rfbEncodingH264 0x48323634
/* Cache & XOR-Zlib - rdv@2002 */
#define rfbEncodingCache 0xFFFF0000
#define rfbEncodingCacheEnable 0xFFFF0001
#define rfbEncodingXOR_Zlib 0xFFFF0002
#define rfbEncodingXORMonoColor_Zlib 0xFFFF0003
#define rfbEncodingXORMultiColor_Zlib 0xFFFF0004
#define rfbEncodingSolidColor 0xFFFF0005
#define rfbEncodingXOREnable 0xFFFF0006
#define rfbEncodingCacheZip 0xFFFF0007
#define rfbEncodingSolMonoZip 0xFFFF0008
#define rfbEncodingUltraZip 0xFFFF0009
/* Xvp pseudo-encoding */
#define rfbEncodingXvp 0xFFFFFECB
22 years ago
/*
* Special encoding numbers:
Add TurboVNC encoding support. TurboVNC is a variant of TightVNC that uses the same client/server protocol (RFB version 3.8t), and thus it is fully cross-compatible with TightVNC and TigerVNC (with one exception, which is noted below.) Both the TightVNC and TurboVNC encoders analyze each rectangle, pick out regions of solid color to send separately, and send the remaining subrectangles using mono, indexed color, JPEG, or raw encoding, depending on the number of colors in the subrectangle. However, TurboVNC uses a fundamentally different selection algorithm to determine the appropriate subencoding to use for each subrectangle. Thus, while it sends a protocol stream that can be decoded by any TightVNC-compatible viewer, the mix of subencoding types in this protocol stream will be different from those generated by a TightVNC server. The research that led to TurboVNC is described in the following report: http://www.virtualgl.org/pmwiki/uploads/About/tighttoturbo.pdf. In summary: 20 RFB captures, representing "common" 2D and 3D application workloads (the 3D workloads were run using VirtualGL), were studied using the TightVNC encoder in isolation. Some of the analysis features in the TightVNC encoder, such as smoothness detection, were found to generate a lot of CPU usage with little or no benefit in compression, so those features were disabled. JPEG encoding was accelerated using libjpeg-turbo (which achieves a 2-4x speedup over plain libjpeg on modern x86 or ARM processors.) Finally, the "palette threshold" (minimum number of colors that the subrectangle must have before it is compressed using JPEG or raw) was adjusted to account for the fact that JPEG encoding is now quite a bit faster (meaning that we can now use it more without a CPU penalty.) TurboVNC has additional optimizations, such as the ability to count colors and encode JPEG images directly from the framebuffer without first translating the pixels into RGB. The TurboVNC encoder compares quite favorably in terms of compression ratio with TightVNC and generally encodes a great deal faster (often an order of magnitude or more.) The version of the TurboVNC encoder included in this patch is roughly equivalent to the one found in version 0.6 of the Unix TurboVNC Server, with a few minor patches integrated from TurboVNC 1.1. TurboVNC 1.0 added multi-threading capabilities, which can be added in later if desired (at the expense of making libvncserver depend on libpthread.) Because TurboVNC uses a fundamentally different mix of subencodings than TightVNC, because it uses the identical protocol (and thus a viewer really has no idea whether it's talking to a TightVNC or TurboVNC server), and because it doesn't support rfbTightPng (and in fact conflicts with it-- see below), the TurboVNC and TightVNC encoders cannot be enabled simultaneously. Compatibility: In *most* cases, a TurboVNC-enabled viewer is fully compatible with a TightVNC server, and vice versa. TurboVNC supports pseudo-encodings for specifying a fine-grained (1-100) quality scale and specifying chrominance subsampling. If a TurboVNC viewer sends those to a TightVNC server, then the TightVNC server ignores them, so the TurboVNC viewer also sends the quality on a 0-9 scale that the TightVNC server can understand. Similarly, the TurboVNC server checks first for fine-grained quality and subsampling pseudo-encodings from the viewer, and failing to receive those, it then checks for the TightVNC 0-9 quality pseudo-encoding. There is one case in which the two systems are not compatible, and that is when a TightVNC or TigerVNC viewer requests compression level 0 without JPEG from a TurboVNC server. For performance reasons, this causes the TurboVNC server to send images directly to the viewer, bypassing Zlib. When the TurboVNC server does this, it also sets bits 7-4 in the compression control byte to rfbTightNoZlib (0x0A), which is unfortunately the same value as rfbTightPng. Older TightVNC viewers that don't handle PNG will assume that the stream is uncompressed but still encapsulated in a Zlib structure, whereas newer PNG-supporting TightVNC viewers will assume that the stream is PNG. In either case, the viewer will probably crash. Since most VNC viewers don't expose compression level 0 in the GUI, this is a relatively rare situation. Description of changes: configure.ac -- Added support for libjpeg-turbo. If passed an argument of --with-turbovnc, configure will now run (or, if cross-compiling, just link) a test program that determines whether the libjpeg library being used is libjpeg-turbo. libjpeg-turbo must be used when building the TurboVNC encoder, because the TurboVNC encoder relies on the libjpeg-turbo colorspace extensions in order to compress images directly out of the framebuffer (which may be, for instance, BGRA rather than RGB.) libjpeg-turbo can optionally be used with the TightVNC encoder as well, but the speedup will only be marginal (the report linked above explains why in more detail, but basically it's because of Amdahl's Law. The TightVNC encoder was designed with the assumption that JPEG had a very high CPU cost, and thus JPEG is used only sparingly.) -- Added a new configure variable, JPEG_LDFLAGS. This is necessitated by the fact that libjpeg-turbo often distributes libjpeg.a and libjpeg.so in /opt/libjpeg-turbo/lib32 or /opt/libjpeg-turbo/lib64, and many people prefer to statically link with it. Thus, more flexibility is needed than is provided by --with-jpeg. If JPEG_LDFLAGS is specified, then it overrides the changes to LDFLAGS enacted by --with-jpeg (but --with-jpeg is still used to set the include path.) The addition of JPEG_LDFLAGS necessitated replacing AC_CHECK_LIB with AC_LINK_IFELSE (because AC_CHECK_LIB automatically sets LIBS to -ljpeg, which is not what we want if we're, for instance, linking statically with libjpeg-turbo.) -- configure does not check for PNG support if TurboVNC encoding is enabled. This prevents the rfbSendRectEncodingTightPng() function from being compiled in, since the TurboVNC encoder doesn't (and can't) support it. common/turbojpeg.c, common/turbojpeg.h -- TurboJPEG is a simple API used to compress and decompress JPEG images in memory. It was originally implemented because it was desirable to use different types of underlying technologies to compress JPEG on different platforms (mediaLib on SPARC, Quicktime on PPC Macs, Intel Performance Primitives, etc.) These days, however, libjpeg-turbo is the only underlying technology used by TurboVNC, so TurboJPEG's purpose is largely just code simplicity and flexibility. Thus, since there is no real need for libvncserver to use any technology other than libjpeg-turbo for compressing JPEG, the TurboJPEG wrapper for libjpeg-turbo has been included in-tree so that libvncserver can be directly linked with libjpeg-turbo. This is convenient because many modern Linux distros (Fedora, Ubuntu, etc.) now ship libjpeg-turbo as their default libjpeg library. libvncserver/rfbserver.c -- Added logic to check for the TurboVNC fine-grained quality level and subsampling encodings and to map Tight (0-9) quality levels to appropriate fine-grained quality level and subsampling values if communicating with a TightVNC/TigerVNC viewer. libvncserver/turbo.c -- TurboVNC encoder (compiled instead of libvncserver/tight.c) rfb/rfb.h -- Added support for the TurboVNC subsampling level rfb/rfbproto.h -- Added constants for the TurboVNC fine quality level and subsampling encodings as well as the rfbTightNoZlib constant and notes on its usage.
11 years ago
* 0xFFFFFD00 .. 0xFFFFFD05 -- subsampling level
* 0xFFFFFE00 .. 0xFFFFFE64 -- fine-grained quality level (0-100 scale)
22 years ago
* 0xFFFFFF00 .. 0xFFFFFF0F -- encoding-specific compression levels;
* 0xFFFFFF10 .. 0xFFFFFF1F -- mouse cursor shape data;
* 0xFFFFFF20 .. 0xFFFFFF2F -- various protocol extensions;
* 0xFFFFFF30 .. 0xFFFFFFDF -- not allocated yet;
* 0xFFFFFFE0 .. 0xFFFFFFEF -- quality level for JPEG compressor;
* 0xFFFFFFF0 .. 0xFFFFFFFF -- cross-encoding compression levels.
*/
Add TurboVNC encoding support. TurboVNC is a variant of TightVNC that uses the same client/server protocol (RFB version 3.8t), and thus it is fully cross-compatible with TightVNC and TigerVNC (with one exception, which is noted below.) Both the TightVNC and TurboVNC encoders analyze each rectangle, pick out regions of solid color to send separately, and send the remaining subrectangles using mono, indexed color, JPEG, or raw encoding, depending on the number of colors in the subrectangle. However, TurboVNC uses a fundamentally different selection algorithm to determine the appropriate subencoding to use for each subrectangle. Thus, while it sends a protocol stream that can be decoded by any TightVNC-compatible viewer, the mix of subencoding types in this protocol stream will be different from those generated by a TightVNC server. The research that led to TurboVNC is described in the following report: http://www.virtualgl.org/pmwiki/uploads/About/tighttoturbo.pdf. In summary: 20 RFB captures, representing "common" 2D and 3D application workloads (the 3D workloads were run using VirtualGL), were studied using the TightVNC encoder in isolation. Some of the analysis features in the TightVNC encoder, such as smoothness detection, were found to generate a lot of CPU usage with little or no benefit in compression, so those features were disabled. JPEG encoding was accelerated using libjpeg-turbo (which achieves a 2-4x speedup over plain libjpeg on modern x86 or ARM processors.) Finally, the "palette threshold" (minimum number of colors that the subrectangle must have before it is compressed using JPEG or raw) was adjusted to account for the fact that JPEG encoding is now quite a bit faster (meaning that we can now use it more without a CPU penalty.) TurboVNC has additional optimizations, such as the ability to count colors and encode JPEG images directly from the framebuffer without first translating the pixels into RGB. The TurboVNC encoder compares quite favorably in terms of compression ratio with TightVNC and generally encodes a great deal faster (often an order of magnitude or more.) The version of the TurboVNC encoder included in this patch is roughly equivalent to the one found in version 0.6 of the Unix TurboVNC Server, with a few minor patches integrated from TurboVNC 1.1. TurboVNC 1.0 added multi-threading capabilities, which can be added in later if desired (at the expense of making libvncserver depend on libpthread.) Because TurboVNC uses a fundamentally different mix of subencodings than TightVNC, because it uses the identical protocol (and thus a viewer really has no idea whether it's talking to a TightVNC or TurboVNC server), and because it doesn't support rfbTightPng (and in fact conflicts with it-- see below), the TurboVNC and TightVNC encoders cannot be enabled simultaneously. Compatibility: In *most* cases, a TurboVNC-enabled viewer is fully compatible with a TightVNC server, and vice versa. TurboVNC supports pseudo-encodings for specifying a fine-grained (1-100) quality scale and specifying chrominance subsampling. If a TurboVNC viewer sends those to a TightVNC server, then the TightVNC server ignores them, so the TurboVNC viewer also sends the quality on a 0-9 scale that the TightVNC server can understand. Similarly, the TurboVNC server checks first for fine-grained quality and subsampling pseudo-encodings from the viewer, and failing to receive those, it then checks for the TightVNC 0-9 quality pseudo-encoding. There is one case in which the two systems are not compatible, and that is when a TightVNC or TigerVNC viewer requests compression level 0 without JPEG from a TurboVNC server. For performance reasons, this causes the TurboVNC server to send images directly to the viewer, bypassing Zlib. When the TurboVNC server does this, it also sets bits 7-4 in the compression control byte to rfbTightNoZlib (0x0A), which is unfortunately the same value as rfbTightPng. Older TightVNC viewers that don't handle PNG will assume that the stream is uncompressed but still encapsulated in a Zlib structure, whereas newer PNG-supporting TightVNC viewers will assume that the stream is PNG. In either case, the viewer will probably crash. Since most VNC viewers don't expose compression level 0 in the GUI, this is a relatively rare situation. Description of changes: configure.ac -- Added support for libjpeg-turbo. If passed an argument of --with-turbovnc, configure will now run (or, if cross-compiling, just link) a test program that determines whether the libjpeg library being used is libjpeg-turbo. libjpeg-turbo must be used when building the TurboVNC encoder, because the TurboVNC encoder relies on the libjpeg-turbo colorspace extensions in order to compress images directly out of the framebuffer (which may be, for instance, BGRA rather than RGB.) libjpeg-turbo can optionally be used with the TightVNC encoder as well, but the speedup will only be marginal (the report linked above explains why in more detail, but basically it's because of Amdahl's Law. The TightVNC encoder was designed with the assumption that JPEG had a very high CPU cost, and thus JPEG is used only sparingly.) -- Added a new configure variable, JPEG_LDFLAGS. This is necessitated by the fact that libjpeg-turbo often distributes libjpeg.a and libjpeg.so in /opt/libjpeg-turbo/lib32 or /opt/libjpeg-turbo/lib64, and many people prefer to statically link with it. Thus, more flexibility is needed than is provided by --with-jpeg. If JPEG_LDFLAGS is specified, then it overrides the changes to LDFLAGS enacted by --with-jpeg (but --with-jpeg is still used to set the include path.) The addition of JPEG_LDFLAGS necessitated replacing AC_CHECK_LIB with AC_LINK_IFELSE (because AC_CHECK_LIB automatically sets LIBS to -ljpeg, which is not what we want if we're, for instance, linking statically with libjpeg-turbo.) -- configure does not check for PNG support if TurboVNC encoding is enabled. This prevents the rfbSendRectEncodingTightPng() function from being compiled in, since the TurboVNC encoder doesn't (and can't) support it. common/turbojpeg.c, common/turbojpeg.h -- TurboJPEG is a simple API used to compress and decompress JPEG images in memory. It was originally implemented because it was desirable to use different types of underlying technologies to compress JPEG on different platforms (mediaLib on SPARC, Quicktime on PPC Macs, Intel Performance Primitives, etc.) These days, however, libjpeg-turbo is the only underlying technology used by TurboVNC, so TurboJPEG's purpose is largely just code simplicity and flexibility. Thus, since there is no real need for libvncserver to use any technology other than libjpeg-turbo for compressing JPEG, the TurboJPEG wrapper for libjpeg-turbo has been included in-tree so that libvncserver can be directly linked with libjpeg-turbo. This is convenient because many modern Linux distros (Fedora, Ubuntu, etc.) now ship libjpeg-turbo as their default libjpeg library. libvncserver/rfbserver.c -- Added logic to check for the TurboVNC fine-grained quality level and subsampling encodings and to map Tight (0-9) quality levels to appropriate fine-grained quality level and subsampling values if communicating with a TightVNC/TigerVNC viewer. libvncserver/turbo.c -- TurboVNC encoder (compiled instead of libvncserver/tight.c) rfb/rfb.h -- Added support for the TurboVNC subsampling level rfb/rfbproto.h -- Added constants for the TurboVNC fine quality level and subsampling encodings as well as the rfbTightNoZlib constant and notes on its usage.
11 years ago
#define rfbEncodingFineQualityLevel0 0xFFFFFE00
#define rfbEncodingFineQualityLevel100 0xFFFFFE64
#define rfbEncodingSubsamp1X 0xFFFFFD00
#define rfbEncodingSubsamp4X 0xFFFFFD01
#define rfbEncodingSubsamp2X 0xFFFFFD02
#define rfbEncodingSubsampGray 0xFFFFFD03
#define rfbEncodingSubsamp8X 0xFFFFFD04
#define rfbEncodingSubsamp16X 0xFFFFFD05
22 years ago
#define rfbEncodingCompressLevel0 0xFFFFFF00
#define rfbEncodingCompressLevel1 0xFFFFFF01
#define rfbEncodingCompressLevel2 0xFFFFFF02
#define rfbEncodingCompressLevel3 0xFFFFFF03
#define rfbEncodingCompressLevel4 0xFFFFFF04
#define rfbEncodingCompressLevel5 0xFFFFFF05
#define rfbEncodingCompressLevel6 0xFFFFFF06
#define rfbEncodingCompressLevel7 0xFFFFFF07
#define rfbEncodingCompressLevel8 0xFFFFFF08
#define rfbEncodingCompressLevel9 0xFFFFFF09
#define rfbEncodingXCursor 0xFFFFFF10
#define rfbEncodingRichCursor 0xFFFFFF11
#define rfbEncodingPointerPos 0xFFFFFF18
22 years ago
#define rfbEncodingLastRect 0xFFFFFF20
#define rfbEncodingNewFBSize 0xFFFFFF21
22 years ago
#define rfbEncodingQualityLevel0 0xFFFFFFE0
#define rfbEncodingQualityLevel1 0xFFFFFFE1
#define rfbEncodingQualityLevel2 0xFFFFFFE2
#define rfbEncodingQualityLevel3 0xFFFFFFE3
#define rfbEncodingQualityLevel4 0xFFFFFFE4
#define rfbEncodingQualityLevel5 0xFFFFFFE5
#define rfbEncodingQualityLevel6 0xFFFFFFE6
#define rfbEncodingQualityLevel7 0xFFFFFFE7
#define rfbEncodingQualityLevel8 0xFFFFFFE8
#define rfbEncodingQualityLevel9 0xFFFFFFE9
/* LibVNCServer additions. We claim 0xFFFE0000 - 0xFFFE00FF */
#define rfbEncodingKeyboardLedState 0xFFFE0000
#define rfbEncodingSupportedMessages 0xFFFE0001
#define rfbEncodingSupportedEncodings 0xFFFE0002
#define rfbEncodingServerIdentity 0xFFFE0003
22 years ago
/*****************************************************************************
*
* Server -> client message definitions
*
*****************************************************************************/
/*-----------------------------------------------------------------------------
* FramebufferUpdate - a block of rectangles to be copied to the framebuffer.
*
* This message consists of a header giving the number of rectangles of pixel
* data followed by the rectangles themselves. The header is padded so that
* together with the type byte it is an exact multiple of 4 bytes (to help
* with alignment of 32-bit pixels):
*/
typedef struct {
uint8_t type; /* always rfbFramebufferUpdate */
uint8_t pad;
uint16_t nRects;
22 years ago
/* followed by nRects rectangles */
} rfbFramebufferUpdateMsg;
#define sz_rfbFramebufferUpdateMsg 4
/*
* Each rectangle of pixel data consists of a header describing the position
* and size of the rectangle and a type word describing the encoding of the
* pixel data, followed finally by the pixel data. Note that if the client has
* not sent a SetEncodings message then it will only receive raw pixel data.
* Also note again that this structure is a multiple of 4 bytes.
*/
typedef struct {
rfbRectangle r;
uint32_t encoding; /* one of the encoding types rfbEncoding... */
22 years ago
} rfbFramebufferUpdateRectHeader;
#define sz_rfbFramebufferUpdateRectHeader (sz_rfbRectangle + 4)
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* Supported Messages Encoding. This encoding does not contain any pixel data.
* Instead, it contains 2 sets of bitflags. These bitflags indicate what messages
* are supported by the server.
* rect->w contains byte count
*/
typedef struct {
uint8_t client2server[32]; /* maximum of 256 message types (256/8)=32 */
uint8_t server2client[32]; /* maximum of 256 message types (256/8)=32 */
} rfbSupportedMessages;
#define sz_rfbSupportedMessages 64
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* Supported Encodings Encoding. This encoding does not contain any pixel data.
* Instead, it contains a list of (uint32_t) Encodings supported by this server.
* rect->w contains byte count
* rect->h contains encoding count
*/
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* Server Identity Encoding. This encoding does not contain any pixel data.
* Instead, it contains a text string containing information about the server.
* ie: "x11vnc: 0.8.1 lastmod: 2006-04-25 (libvncserver 0.9pre)\0"
* rect->w contains byte count
*/
22 years ago
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* Raw Encoding. Pixels are sent in top-to-bottom scanline order,
* left-to-right within a scanline with no padding in between.
*/
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* KeyboardLedState Encoding. The X coordinate contains the Locked Modifiers
* so that a remote troubleshooter can identify that the users 'Caps Lock'
* is set... (It helps a *lot* when the users are untrained)
*/
#define rfbKeyboardMaskShift 1
#define rfbKeyboardMaskCapsLock 2
#define rfbKeyboardMaskControl 4
#define rfbKeyboardMaskAlt 8
#define rfbKeyboardMaskMeta 16
#define rfbKeyboardMaskSuper 32
#define rfbKeyboardMaskHyper 64
#define rfbKeyboardMaskNumLock 128
#define rfbKeyboardMaskScrollLock 256
#define rfbKeyboardMaskAltGraph 512
22 years ago
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* CopyRect Encoding. The pixels are specified simply by the x and y position
* of the source rectangle.
*/
typedef struct {
uint16_t srcX;
uint16_t srcY;
22 years ago
} rfbCopyRect;
#define sz_rfbCopyRect 4
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* RRE - Rise-and-Run-length Encoding. We have an rfbRREHeader structure
* giving the number of subrectangles following. Finally the data follows in
* the form [<bgpixel><subrect><subrect>...] where each <subrect> is
* [<pixel><rfbRectangle>].
*/
typedef struct {
uint32_t nSubrects;
22 years ago
} rfbRREHeader;
#define sz_rfbRREHeader 4
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* CoRRE - Compact RRE Encoding. We have an rfbRREHeader structure giving
* the number of subrectangles following. Finally the data follows in the form
* [<bgpixel><subrect><subrect>...] where each <subrect> is
* [<pixel><rfbCoRRERectangle>]. This means that
* the whole rectangle must be at most 255x255 pixels.
*/
typedef struct {
uint8_t x;
uint8_t y;
uint8_t w;
uint8_t h;
22 years ago
} rfbCoRRERectangle;
#define sz_rfbCoRRERectangle 4
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* Hextile Encoding. The rectangle is divided up into "tiles" of 16x16 pixels,
* starting at the top left going in left-to-right, top-to-bottom order. If
* the width of the rectangle is not an exact multiple of 16 then the width of
* the last tile in each row will be correspondingly smaller. Similarly if the
* height is not an exact multiple of 16 then the height of each tile in the
* final row will also be smaller. Each tile begins with a "subencoding" type
* byte, which is a mask made up of a number of bits. If the Raw bit is set
* then the other bits are irrelevant; w*h pixel values follow (where w and h
* are the width and height of the tile). Otherwise the tile is encoded in a
* similar way to RRE, except that the position and size of each subrectangle
* can be specified in just two bytes. The other bits in the mask are as
* follows:
*
* BackgroundSpecified - if set, a pixel value follows which specifies
* the background colour for this tile. The first non-raw tile in a
* rectangle must have this bit set. If this bit isn't set then the
* background is the same as the last tile.
*
* ForegroundSpecified - if set, a pixel value follows which specifies
* the foreground colour to be used for all subrectangles in this tile.
* If this bit is set then the SubrectsColoured bit must be zero.
*
* AnySubrects - if set, a single byte follows giving the number of
* subrectangles following. If not set, there are no subrectangles (i.e.
* the whole tile is just solid background colour).
*
* SubrectsColoured - if set then each subrectangle is preceded by a pixel
* value giving the colour of that subrectangle. If not set, all
* subrectangles are the same colour, the foreground colour; if the
* ForegroundSpecified bit wasn't set then the foreground is the same as
* the last tile.
*
* The position and size of each subrectangle is specified in two bytes. The
* Pack macros below can be used to generate the two bytes from x, y, w, h,
* and the Extract macros can be used to extract the x, y, w, h values from
* the two bytes.
*/
#define rfbHextileRaw (1 << 0)
#define rfbHextileBackgroundSpecified (1 << 1)
#define rfbHextileForegroundSpecified (1 << 2)
#define rfbHextileAnySubrects (1 << 3)
#define rfbHextileSubrectsColoured (1 << 4)
#define rfbHextilePackXY(x,y) (((x) << 4) | (y))
#define rfbHextilePackWH(w,h) ((((w)-1) << 4) | ((h)-1))
#define rfbHextileExtractX(byte) ((byte) >> 4)
#define rfbHextileExtractY(byte) ((byte) & 0xf)
#define rfbHextileExtractW(byte) (((byte) >> 4) + 1)
#define rfbHextileExtractH(byte) (((byte) & 0xf) + 1)
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* zlib - zlib compressed Encoding. We have an rfbZlibHeader structure
* giving the number of bytes following. Finally the data follows is
* zlib compressed version of the raw pixel data as negotiated.
* (NOTE: also used by Ultra Encoding)
22 years ago
*/
typedef struct {
uint32_t nBytes;
22 years ago
} rfbZlibHeader;
#define sz_rfbZlibHeader 4
#ifdef LIBVNCSERVER_HAVE_LIBZ
22 years ago
/*- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
* Tight and TightPng Encoding.
*
*-- TightPng is like Tight but basic compression is not used, instead PNG
* data is sent.
*
*-- The first byte of each Tight-encoded rectangle is a "compression control
* byte". Its format is as follows (bit 0 is the least significant one):
*
* bit 0: if 1, then compression stream 0 should be reset;
* bit 1: if 1, then compression stream 1 should be reset;
* bit 2: if 1, then compression stream 2 should be reset;
* bit 3: if 1, then compression stream 3 should be reset;
* bits 7-4: if 1000 (0x08), then the compression type is "fill",
* if 1001 (0x09), then the compression type is "jpeg",
Replace TightVNC encoder with TurboVNC encoder. This patch is the result of further research and discussion that revealed the following: -- TightPng encoding and the rfbTightNoZlib extension need not conflict. Since TightPng is a separate encoding type, not supported by TurboVNC-compatible viewers, then the rfbTightNoZlib extension can be used solely whenever the encoding type is Tight and disabled with the encoding type is TightPng. -- In the TightVNC encoder, compression levels above 5 are basically useless. On the set of 20 low-level datasets that were used to design the TurboVNC encoder (these include the eight 2D application captures that were also used when designing the TightVNC encoder, as well as 12 3D application captures provided by the VirtualGL Project-- see http://www.virtualgl.org/pmwiki/uploads/About/tighttoturbo.pdf), moving from Compression Level (CL) 5 to CL 9 in the TightVNC encoder did not increase the compression ratio of any datasets more than 10%, and the compression ratio only increased by more than 5% on four of them. The compression ratio actually decreased a few percent on five of them. In exchange for this paltry increase in compression ratio, the CPU usage, on average, went up by a factor of 5. Thus, for all intents and purposes, TightVNC CL 5 provides the "best useful compression" for that encoder. -- TurboVNC's best compression level (CL 2) compresses 3D and video workloads significantly more "tightly" than TightVNC CL 5 (~70% better, in the aggregate) but does not quite achieve the same level of compression with 2D workloads (~20% worse, in the aggregate.) This decrease in compression ratio may or may not be noticeable, since many of the datasets it affects are not performance-critical (such as the console output of a compilation, etc.) However, for peace of mind, it was still desirable to have a mode that compressed with equal "tightness" to TightVNC CL 5, since we proposed to replace that encoder entirely. -- A new mode was discovered in the TurboVNC encoder that produces, in the aggregate, similar compression ratios on 2D datasets as TightVNC CL 5. That new mode involves using Zlib level 7 (the same level used by TightVNC CL 5) but setting the "palette threshold" to 256, so that indexed color encoding is used whenever possible. This mode reduces bandwidth only marginally (typically 10-20%) relative to TurboVNC CL 2 on low-color workloads, in exchange for nearly doubling CPU usage, and it does not benefit high-color workloads at all (since those are usually encoded with JPEG.) However, it provides a means of reproducing the same "tightness" as the TightVNC encoder on 2D workloads without sacrificing any compression for 3D/video workloads, and without using any more CPU time than necessary. -- The TurboVNC encoder still performs as well or better than the TightVNC encoder when plain libjpeg is used instead of libjpeg-turbo. Specific notes follow: common/turbojpeg.c common/turbojpeg.h: Added code to emulate the libjpeg-turbo colorspace extensions, so that the TurboJPEG wrapper can be used with plain libjpeg as well. This required updating the TurboJPEG wrapper to the latest code from libjpeg-turbo 1.2.0, mainly because the TurboJPEG 1.2 API handles pixel formats in a much cleaner way, which made the conversion code easier to write. It also eases the maintenance to have the wrapper synced as much as possible with the upstream code base (so I can merge any relevant bug fixes that are discovered upstream.) The libvncserver version of the TurboJPEG wrapper is a "lite" version, containing only the JPEG compression/decompression code and not the lossless transform, YUV encoding/decoding, and dynamic buffer allocation features from TurboJPEG 1.2. configure.ac: Removed the --with-turbovnc option. configure still checks for the presence of libjpeg-turbo, but only for the purposes of printing a performance warning if it isn't available. rfb/rfb.h: Fix a bug introduced with the initial TurboVNC encoder patch. We cannot use tightQualityLevel for the TurboVNC 1-100 quality level, because tightQualityLevel is also used by ZRLE. Thus, a new parameter (turboQualityLevel) was created. rfb/rfbproto.h: Remove TurboVNC-specific #ifdefs and language libvncserver/rfbserver.c: Remove TurboVNC-specific #ifdefs. Fix afore-mentioned tightQualityLevel bug. libvncserver/tight.c: Replaced the TightVNC encoder with the TurboVNC encoder. Relative to the initial TurboVNC encoder patch, this patch also: -- Adds TightPng support to the TurboVNC encoder -- Adds the afore-mentioned low-bandwidth mode, which is mapped externally to Compression Level 9 test/*: Included TJUnitTest (a regression test for the TurboJPEG wrapper) as well as TJBench (a benchmark for same.) These are useful for ensuring that the wrapper still functions correctly and performantly if it needs to be modified for whatever reason. Both of these programs are derived from libjpeg-turbo 1.2.0. As with the TurboJPEG wrapper, they do not contain the more advanced features of TurboJPEG 1.2, such as YUV encoding/decoding and lossless transforms.
11 years ago
* (Tight only) if 1010 (0x0A), then the compression type is
* "basic" and no Zlib compression was used,
* (Tight only) if 1110 (0x0E), then the compression type is
* "basic", no Zlib compression was used, and a "filter id" byte
* follows this byte,
* (TightPng only) if 1010 (0x0A), then the compression type is
* "png",
Add TurboVNC encoding support. TurboVNC is a variant of TightVNC that uses the same client/server protocol (RFB version 3.8t), and thus it is fully cross-compatible with TightVNC and TigerVNC (with one exception, which is noted below.) Both the TightVNC and TurboVNC encoders analyze each rectangle, pick out regions of solid color to send separately, and send the remaining subrectangles using mono, indexed color, JPEG, or raw encoding, depending on the number of colors in the subrectangle. However, TurboVNC uses a fundamentally different selection algorithm to determine the appropriate subencoding to use for each subrectangle. Thus, while it sends a protocol stream that can be decoded by any TightVNC-compatible viewer, the mix of subencoding types in this protocol stream will be different from those generated by a TightVNC server. The research that led to TurboVNC is described in the following report: http://www.virtualgl.org/pmwiki/uploads/About/tighttoturbo.pdf. In summary: 20 RFB captures, representing "common" 2D and 3D application workloads (the 3D workloads were run using VirtualGL), were studied using the TightVNC encoder in isolation. Some of the analysis features in the TightVNC encoder, such as smoothness detection, were found to generate a lot of CPU usage with little or no benefit in compression, so those features were disabled. JPEG encoding was accelerated using libjpeg-turbo (which achieves a 2-4x speedup over plain libjpeg on modern x86 or ARM processors.) Finally, the "palette threshold" (minimum number of colors that the subrectangle must have before it is compressed using JPEG or raw) was adjusted to account for the fact that JPEG encoding is now quite a bit faster (meaning that we can now use it more without a CPU penalty.) TurboVNC has additional optimizations, such as the ability to count colors and encode JPEG images directly from the framebuffer without first translating the pixels into RGB. The TurboVNC encoder compares quite favorably in terms of compression ratio with TightVNC and generally encodes a great deal faster (often an order of magnitude or more.) The version of the TurboVNC encoder included in this patch is roughly equivalent to the one found in version 0.6 of the Unix TurboVNC Server, with a few minor patches integrated from TurboVNC 1.1. TurboVNC 1.0 added multi-threading capabilities, which can be added in later if desired (at the expense of making libvncserver depend on libpthread.) Because TurboVNC uses a fundamentally different mix of subencodings than TightVNC, because it uses the identical protocol (and thus a viewer really has no idea whether it's talking to a TightVNC or TurboVNC server), and because it doesn't support rfbTightPng (and in fact conflicts with it-- see below), the TurboVNC and TightVNC encoders cannot be enabled simultaneously. Compatibility: In *most* cases, a TurboVNC-enabled viewer is fully compatible with a TightVNC server, and vice versa. TurboVNC supports pseudo-encodings for specifying a fine-grained (1-100) quality scale and specifying chrominance subsampling. If a TurboVNC viewer sends those to a TightVNC server, then the TightVNC server ignores them, so the TurboVNC viewer also sends the quality on a 0-9 scale that the TightVNC server can understand. Similarly, the TurboVNC server checks first for fine-grained quality and subsampling pseudo-encodings from the viewer, and failing to receive those, it then checks for the TightVNC 0-9 quality pseudo-encoding. There is one case in which the two systems are not compatible, and that is when a TightVNC or TigerVNC viewer requests compression level 0 without JPEG from a TurboVNC server. For performance reasons, this causes the TurboVNC server to send images directly to the viewer, bypassing Zlib. When the TurboVNC server does this, it also sets bits 7-4 in the compression control byte to rfbTightNoZlib (0x0A), which is unfortunately the same value as rfbTightPng. Older TightVNC viewers that don't handle PNG will assume that the stream is uncompressed but still encapsulated in a Zlib structure, whereas newer PNG-supporting TightVNC viewers will assume that the stream is PNG. In either case, the viewer will probably crash. Since most VNC viewers don't expose compression level 0 in the GUI, this is a relatively rare situation. Description of changes: configure.ac -- Added support for libjpeg-turbo. If passed an argument of --with-turbovnc, configure will now run (or, if cross-compiling, just link) a test program that determines whether the libjpeg library being used is libjpeg-turbo. libjpeg-turbo must be used when building the TurboVNC encoder, because the TurboVNC encoder relies on the libjpeg-turbo colorspace extensions in order to compress images directly out of the framebuffer (which may be, for instance, BGRA rather than RGB.) libjpeg-turbo can optionally be used with the TightVNC encoder as well, but the speedup will only be marginal (the report linked above explains why in more detail, but basically it's because of Amdahl's Law. The TightVNC encoder was designed with the assumption that JPEG had a very high CPU cost, and thus JPEG is used only sparingly.) -- Added a new configure variable, JPEG_LDFLAGS. This is necessitated by the fact that libjpeg-turbo often distributes libjpeg.a and libjpeg.so in /opt/libjpeg-turbo/lib32 or /opt/libjpeg-turbo/lib64, and many people prefer to statically link with it. Thus, more flexibility is needed than is provided by --with-jpeg. If JPEG_LDFLAGS is specified, then it overrides the changes to LDFLAGS enacted by --with-jpeg (but --with-jpeg is still used to set the include path.) The addition of JPEG_LDFLAGS necessitated replacing AC_CHECK_LIB with AC_LINK_IFELSE (because AC_CHECK_LIB automatically sets LIBS to -ljpeg, which is not what we want if we're, for instance, linking statically with libjpeg-turbo.) -- configure does not check for PNG support if TurboVNC encoding is enabled. This prevents the rfbSendRectEncodingTightPng() function from being compiled in, since the TurboVNC encoder doesn't (and can't) support it. common/turbojpeg.c, common/turbojpeg.h -- TurboJPEG is a simple API used to compress and decompress JPEG images in memory. It was originally implemented because it was desirable to use different types of underlying technologies to compress JPEG on different platforms (mediaLib on SPARC, Quicktime on PPC Macs, Intel Performance Primitives, etc.) These days, however, libjpeg-turbo is the only underlying technology used by TurboVNC, so TurboJPEG's purpose is largely just code simplicity and flexibility. Thus, since there is no real need for libvncserver to use any technology other than libjpeg-turbo for compressing JPEG, the TurboJPEG wrapper for libjpeg-turbo has been included in-tree so that libvncserver can be directly linked with libjpeg-turbo. This is convenient because many modern Linux distros (Fedora, Ubuntu, etc.) now ship libjpeg-turbo as their default libjpeg library. libvncserver/rfbserver.c -- Added logic to check for the TurboVNC fine-grained quality level and subsampling encodings and to map Tight (0-9) quality levels to appropriate fine-grained quality level and subsampling values if communicating with a TightVNC/TigerVNC viewer. libvncserver/turbo.c -- TurboVNC encoder (compiled instead of libvncserver/tight.c) rfb/rfb.h -- Added support for the TurboVNC subsampling level rfb/rfbproto.h -- Added constants for the TurboVNC fine quality level and subsampling encodings as well as the rfbTightNoZlib constant and notes on its usage.
11 years ago
* if 0xxx, then the compression type is "basic" and Zlib
* compression was used,
* values greater than 1010 are not valid.
*
Add TurboVNC encoding support. TurboVNC is a variant of TightVNC that uses the same client/server protocol (RFB version 3.8t), and thus it is fully cross-compatible with TightVNC and TigerVNC (with one exception, which is noted below.) Both the TightVNC and TurboVNC encoders analyze each rectangle, pick out regions of solid color to send separately, and send the remaining subrectangles using mono, indexed color, JPEG, or raw encoding, depending on the number of colors in the subrectangle. However, TurboVNC uses a fundamentally different selection algorithm to determine the appropriate subencoding to use for each subrectangle. Thus, while it sends a protocol stream that can be decoded by any TightVNC-compatible viewer, the mix of subencoding types in this protocol stream will be different from those generated by a TightVNC server. The research that led to TurboVNC is described in the following report: http://www.virtualgl.org/pmwiki/uploads/About/tighttoturbo.pdf. In summary: 20 RFB captures, representing "common" 2D and 3D application workloads (the 3D workloads were run using VirtualGL), were studied using the TightVNC encoder in isolation. Some of the analysis features in the TightVNC encoder, such as smoothness detection, were found to generate a lot of CPU usage with little or no benefit in compression, so those features were disabled. JPEG encoding was accelerated using libjpeg-turbo (which achieves a 2-4x speedup over plain libjpeg on modern x86 or ARM processors.) Finally, the "palette threshold" (minimum number of colors that the subrectangle must have before it is compressed using JPEG or raw) was adjusted to account for the fact that JPEG encoding is now quite a bit faster (meaning that we can now use it more without a CPU penalty.) TurboVNC has additional optimizations, such as the ability to count colors and encode JPEG images directly from the framebuffer without first translating the pixels into RGB. The TurboVNC encoder compares quite favorably in terms of compression ratio with TightVNC and generally encodes a great deal faster (often an order of magnitude or more.) The version of the TurboVNC encoder included in this patch is roughly equivalent to the one found in version 0.6 of the Unix TurboVNC Server, with a few minor patches integrated from TurboVNC 1.1. TurboVNC 1.0 added multi-threading capabilities, which can be added in later if desired (at the expense of making libvncserver depend on libpthread.) Because TurboVNC uses a fundamentally different mix of subencodings than TightVNC, because it uses the identical protocol (and thus a viewer really has no idea whether it's talking to a TightVNC or TurboVNC server), and because it doesn't support rfbTightPng (and in fact conflicts with it-- see below), the TurboVNC and TightVNC encoders cannot be enabled simultaneously. Compatibility: In *most* cases, a TurboVNC-enabled viewer is fully compatible with a TightVNC server, and vice versa. TurboVNC supports pseudo-encodings for specifying a fine-grained (1-100) quality scale and specifying chrominance subsampling. If a TurboVNC viewer sends those to a TightVNC server, then the TightVNC server ignores them, so the TurboVNC viewer also sends the quality on a 0-9 scale that the TightVNC server can understand. Similarly, the TurboVNC server checks first for fine-grained quality and subsampling pseudo-encodings from the viewer, and failing to receive those, it then checks for the TightVNC 0-9 quality pseudo-encoding. There is one case in which the two systems are not compatible, and that is when a TightVNC or TigerVNC viewer requests compression level 0 without JPEG from a TurboVNC server. For performance reasons, this causes the TurboVNC server to send images directly to the viewer, bypassing Zlib. When the TurboVNC server does this, it also sets bits 7-4 in the compression control byte to rfbTightNoZlib (0x0A), which is unfortunately the same value as rfbTightPng. Older TightVNC viewers that don't handle PNG will assume that the stream is uncompressed but still encapsulated in a Zlib structure, whereas newer PNG-supporting TightVNC viewers will assume that the stream is PNG. In either case, the viewer will probably crash. Since most VNC viewers don't expose compression level 0 in the GUI, this is a relatively rare situation. Description of changes: configure.ac -- Added support for libjpeg-turbo. If passed an argument of --with-turbovnc, configure will now run (or, if cross-compiling, just link) a test program that determines whether the libjpeg library being used is libjpeg-turbo. libjpeg-turbo must be used when building the TurboVNC encoder, because the TurboVNC encoder relies on the libjpeg-turbo colorspace extensions in order to compress images directly out of the framebuffer (which may be, for instance, BGRA rather than RGB.) libjpeg-turbo can optionally be used with the TightVNC encoder as well, but the speedup will only be marginal (the report linked above explains why in more detail, but basically it's because of Amdahl's Law. The TightVNC encoder was designed with the assumption that JPEG had a very high CPU cost, and thus JPEG is used only sparingly.) -- Added a new configure variable, JPEG_LDFLAGS. This is necessitated by the fact that libjpeg-turbo often distributes libjpeg.a and libjpeg.so in /opt/libjpeg-turbo/lib32 or /opt/libjpeg-turbo/lib64, and many people prefer to statically link with it. Thus, more flexibility is needed than is provided by --with-jpeg. If JPEG_LDFLAGS is specified, then it overrides the changes to LDFLAGS enacted by --with-jpeg (but --with-jpeg is still used to set the include path.) The addition of JPEG_LDFLAGS necessitated replacing AC_CHECK_LIB with AC_LINK_IFELSE (because AC_CHECK_LIB automatically sets LIBS to -ljpeg, which is not what we want if we're, for instance, linking statically with libjpeg-turbo.) -- configure does not check for PNG support if TurboVNC encoding is enabled. This prevents the rfbSendRectEncodingTightPng() function from being compiled in, since the TurboVNC encoder doesn't (and can't) support it. common/turbojpeg.c, common/turbojpeg.h -- TurboJPEG is a simple API used to compress and decompress JPEG images in memory. It was originally implemented because it was desirable to use different types of underlying technologies to compress JPEG on different platforms (mediaLib on SPARC, Quicktime on PPC Macs, Intel Performance Primitives, etc.) These days, however, libjpeg-turbo is the only underlying technology used by TurboVNC, so TurboJPEG's purpose is largely just code simplicity and flexibility. Thus, since there is no real need for libvncserver to use any technology other than libjpeg-turbo for compressing JPEG, the TurboJPEG wrapper for libjpeg-turbo has been included in-tree so that libvncserver can be directly linked with libjpeg-turbo. This is convenient because many modern Linux distros (Fedora, Ubuntu, etc.) now ship libjpeg-turbo as their default libjpeg library. libvncserver/rfbserver.c -- Added logic to check for the TurboVNC fine-grained quality level and subsampling encodings and to map Tight (0-9) quality levels to appropriate fine-grained quality level and subsampling values if communicating with a TightVNC/TigerVNC viewer. libvncserver/turbo.c -- TurboVNC encoder (compiled instead of libvncserver/tight.c) rfb/rfb.h -- Added support for the TurboVNC subsampling level rfb/rfbproto.h -- Added constants for the TurboVNC fine quality level and subsampling encodings as well as the rfbTightNoZlib constant and notes on its usage.
11 years ago
* If the compression type is "basic" and Zlib compression was used, then bits
* 6..4 of the compression control byte (those xxx in 0xxx) specify the
* following:
*
* bits 5-4: decimal representation is the index of a particular zlib
* stream which should be used for decompressing the data;
* bit 6: if 1, then a "filter id" byte is following this byte.
*
*-- The data that follows after the compression control byte described
* above depends on the compression type ("fill", "jpeg", "png" or "basic").
*
*-- If the compression type is "fill", then the only pixel value follows, in
* client pixel format (see NOTE 1). This value applies to all pixels of the
* rectangle.
*
*-- If the compression type is "jpeg" or "png", the following data stream
* looks like this:
*
* 1..3 bytes: data size (N) in compact representation;
* N bytes: JPEG or PNG image.
*
* Data size is compactly represented in one, two or three bytes, according
* to the following scheme:
*
* 0xxxxxxx (for values 0..127)
* 1xxxxxxx 0yyyyyyy (for values 128..16383)
* 1xxxxxxx 1yyyyyyy zzzzzzzz (for values 16384..4194303)
*
* Here each character denotes one bit, xxxxxxx are the least significant 7
* bits of the value (bits 0-6), yyyyyyy are bits 7-13, and zzzzzzzz are the
* most significant 8 bits (bits 14-21). For example, decimal value 10000
* should be represented as two bytes: binary 10010000 01001110, or
* hexadecimal 90 4E.
*
*-- If the compression type is "basic" and bit 6 of the compression control
* byte was set to 1, then the next (second) byte specifies "filter id" which
* tells the decoder what filter type was used by the encoder to pre-process
* pixel data before the compression. The "filter id" byte can be one of the
* following:
*
* 0: no filter ("copy" filter);
* 1: "palette" filter;
* 2: "gradient" filter.
*
*-- If bit 6 of the compression control byte is set to 0 (no "filter id"
* byte), or if the filter id is 0, then raw pixel values in the client
* format (see NOTE 1) will be compressed. See below details on the
* compression.
*
*-- The "gradient" filter pre-processes pixel data with a simple algorithm
* which converts each color component to a difference between a "predicted"
* intensity and the actual intensity. Such a technique does not affect
* uncompressed data size, but helps to compress photo-like images better.
* Pseudo-code for converting intensities to differences is the following:
*
* P[i,j] := V[i-1,j] + V[i,j-1] - V[i-1,j-1];
* if (P[i,j] < 0) then P[i,j] := 0;
* if (P[i,j] > MAX) then P[i,j] := MAX;
* D[i,j] := V[i,j] - P[i,j];
*
* Here V[i,j] is the intensity of a color component for a pixel at
* coordinates (i,j). MAX is the maximum value of intensity for a color
* component.
*
*-- The "palette" filter converts true-color pixel data to indexed colors
* and a palette which can consist of 2..256 colors. If the number of colors
* is 2, then each pixel is encoded in 1 bit, otherwise 8 bits is used to
* encode one pixel. 1-bit encoding is performed such way that the most
* significant bits correspond to the leftmost pixels, and each raw of pixels
* is aligned to the byte boundary. When "palette" filter is used, the
* palette is sent before the pixel data. The palette begins with an unsigned
* byte which value is the number of colors in the palette minus 1 (i.e. 1
* means 2 colors, 255 means 256 colors in the palette). Then follows the
* palette itself which consist of pixel values in client pixel format (see
* NOTE 1).
*
*-- The pixel data is compressed using the zlib library. But if the data
* size after applying the filter but before the compression is less then 12,
* then the data is sent as is, uncompressed. Four separate zlib streams
* (0..3) can be used and the decoder should read the actual stream id from
* the compression control byte (see NOTE 2).
*
* If the compression is not used, then the pixel data is sent as is,
* otherwise the data stream looks like this:
*
* 1..3 bytes: data size (N) in compact representation;
* N bytes: zlib-compressed data.
*
* Data size is compactly represented in one, two or three bytes, just like
* in the "jpeg" compression method (see above).
*
*-- NOTE 1. If the color depth is 24, and all three color components are
* 8-bit wide, then one pixel in Tight encoding is always represented by
* three bytes, where the first byte is red component, the second byte is
* green component, and the third byte is blue component of the pixel color
* value. This applies to colors in palettes as well.
*
*-- NOTE 2. The decoder must reset compression streams' states before
* decoding the rectangle, if some of bits 0,1,2,3 in the compression control
* byte are set to 1. Note that the decoder must reset zlib streams even if
* the compression type is "fill", "jpeg" or "png".
*
*-- NOTE 3. The "gradient" filter and "jpeg" compression may be used only
* when bits-per-pixel value is either 16 or 32, not 8.
*
*-- NOTE 4. The width of any Tight-encoded rectangle cannot exceed 2048
* pixels. If a rectangle is wider, it must be split into several rectangles
* and each one should be encoded separately.
*
22 years ago
*/
#define rfbT