《从零开始搭建游戏服务器》 网络数据压缩——Zlib算法

版权声明:本文为博主原创文章,遵循 CC 4.0 by-sa 版权协议,转载请附上原文出处链接和本声明。
本文链接:https://linsh-tech.blog.csdn.net/article/details/72920907

前言:

关于压缩算法,其实有不少,例如:ZIPRARbzip2等,这里我们举例使用的Zlib相较于前者都要简单一些,与ZIPRAR等归档算法不同,它与bzip2更为接近。那么,下面我们就来尝试一下Zlip在C#和Java中的实现和数据交互:C#中加密Java中解密;又或者是Java中加密C#中解密,这也体现了Zlib的跨平台特性。

C#方面:

通常是指客户端,例如Unity中,可以使用此压缩方法来压缩网络通信的字节数据,服务器接收到数据之后再进行解压,这样可以节约网络通信的流量损耗。

1.下载:

其实很简单,直接查找第三方类库DotNetZipLib-DevKit-v1.9,下载其中的Ionic.Zip.dll库文件,其目录为DotNetZipLib-DevKit-v1.9\zip-v1.9\Release\Ionic.Zip.dll,然后引入到项目中,其实也可以单独下载.net版本的Zlib库文件,这里使用到Ionic.Zip库是因为其中继承了几种压缩算法,为了方便后续使用,导入到应用项目中,可以查看其结构目录:

Ionic.Zip
--Ionic
--Ionic.BZip2
--Ionic.Crc
--Ionic.Zip
--Ionic.Zlib

这里我们使用到的就是Ionic.Zlib这种压缩算法,这里我们以压缩byte[]为例,具体的压缩和解压步骤:

//引入库
using Ionic.Zlib;

2.压缩:

//压缩字节数组
byte[] newData = ZlibStream.CompressBuffer(oldData);

3.解压:

//解压
byte[] oldData = ZlibStream.UncompressBuffer(newData);

Java版的Zlib:

通常是指服务器,在JDK中的java.util.zip中就包含了内置的Zlib实现,我们只需要做简单的封装即可实现压缩和解压的功能,这里我们封装一下压缩和解压byte[]字节数组的方法在工具类 ZlibUtil 中:

1.压缩:

    /**
     * 压缩
     * @param data 待压缩数据
     * @return byte[] 压缩后的数据
     */
    public static byte[] compress(byte[] data) {  
        byte[] output = new byte[0];  

        Deflater compresser = new Deflater();  

        compresser.reset();  
        compresser.setInput(data);  
        compresser.finish();  
        ByteArrayOutputStream bos = new ByteArrayOutputStream(data.length);  
        try {  
            byte[] buf = new byte[1024];  
            while (!compresser.finished()) {  
                int i = compresser.deflate(buf);  
                bos.write(buf, 0, i);  
            }  
            output = bos.toByteArray();  
        } catch (Exception e) {  
            output = data;  
            e.printStackTrace();  
        } finally {  
            try {  
                bos.close();  
            } catch (IOException e) {  
                e.printStackTrace();  
            }  
        }  
        compresser.end();  
        return output;  
    }  

2.解压:

    /** 
     * 解压
     * @param data      待压缩的数据 
     * @return byte[]   解压缩后的数据 
     */  
    public static byte[] decompress(byte[] data) {  
        byte[] output = new byte[0];  

        Inflater decompresser = new Inflater();  
        decompresser.reset();  
        decompresser.setInput(data);  

        ByteArrayOutputStream o = new ByteArrayOutputStream(data.length);  
        try {  
            byte[] buf = new byte[1024];  
            while (!decompresser.finished()) {  
                int i = decompresser.inflate(buf);  
                o.write(buf, 0, i);  
            }  
            output = o.toByteArray();  
        } catch (Exception e) {  
            output = data;  
            e.printStackTrace();  
        } finally {  
            try {  
                o.close();  
            } catch (IOException e) {  
                e.printStackTrace();  
            }  
        }  

        decompresser.end();  
        return output;  
    }

3.测试:

        System.err.println("字节压缩/解压缩测试");  
        String inputStr = "linshuhe.tech;linshuhe.tech;linshuhe.tech;linshuhe.tech;";  
        System.err.println("输入字符串:\t" + inputStr);  
        byte[] input = inputStr.getBytes();  
        System.err.println("输入字节长度:\t" + input.length);
        byte[] data = ZlibUtil.compress(input);  
        System.err.println("压缩后字节长度:\t" + data.length);
        byte[] output = ZlibUtil.decompress(data);  
        System.err.println("解压缩后字节长度:\t" + output.length);  
        String outputStr = new String(output);  
        System.err.println("输出字符串:\t" + outputStr);

运行代码输出的测试结果如下:

字节压缩/解压缩测试
输入字符串:  linshuhe.tech;linshuhe.tech;linshuhe.tech;linshuhe.tech;
输入字节长度: 56
压缩后字节长度:    25
解压缩后字节长度:   56
输出字符串:  linshuhe.tech;linshuhe.tech;linshuhe.tech;linshuhe.tech;

题外话:

Java中除了Zlib这种简单的压缩算法之外,还有其他几种常用的较复杂的压缩算法:

名称 实现方式 性能
JDK GZIP java.util.zip.GZIPInputStream/GZIPOutputStream便可实现 压缩比高,速度慢,压缩后的数据适合长期使用
JDK deflate java.util.zip.DeflaterOutputStream/InflaterInputStream可实现 可指定算法的压缩级别,0(不压缩)、1(快速压缩)到9(慢速压缩)
LZ4压缩算法 https://github.com/lz4/lz4/https://github.com/Cyan4973/lz4.githttps://github.com/lz4/lz4.git 对比几种压缩算法里面压缩速度最快的
Snappy Google开发的 速度和压缩比都相对较优

速度对比结果:
Snappy要慢于LZ4(快速压缩),并且压缩后的文件要更大。相反,LZ4(高压缩比)要慢于级别1到4的deflate,而输出文件的大小即便和级别1的deflate相比也要大上不少。因此如果需要进行“实时压缩”的话我肯定会在LZ4(快速)的JNI实现或者是级别1的deflate中进行选择。当然如果你的公司不允许使用第三方库的话你也只能使用deflate了。

更多关于Zlib的内容参考:Zlib官网


参考资料:

展开阅读全文

C++Builder6操作Zlib进行数据压缩

07-30

今天试了一下开源数据压缩Zlib在C++Builder6中的使用,有些问题,向大家请教一下。将详细步骤贴在下面(帖子较长,给各位添麻烦了~~~不好意思):rnrn1、在网站http://www.zlib.net/中下载了Zlib.dll,上面提供了四个,我下载的是:rnrn“zlib compiled DLL, version 1.2.3, zipfile format (79K, MD5 checksum cc7fa97f9c19386bb701acc79d0abbca)”rnrn2、解压缩,得到文件夹及文件为:include文件夹(包括zconf.h,zlib.h文件),lib文件夹(包括zdll.exp,zdll.lib,zlib.def文件),test文件夹(包括example_d.exe,minigzip_exe,testzlib_d.exe,untgz_d.exe文件),以及四个文件DLL_FAQ.txt,README.txt,USAGE.txt和zlib1.dll文件。rnrn3、利用C++Builder6的implib.exe将步骤2中的zlib1.dll转换为zlib.librnrn4、将步骤2中的include文件夹中的zconf.h,zlib.h文件放到CB的安装目录“..\CBuilder6\Include\”下,自己建立了一个Zlib文件夹存放,便于查找,即最后的存放目录为“..\CBuilder6\Include\Zlib\zconf.h”以及“..\CBuilder6\Include\Zlib\zlib.h”。同理,将步骤3中生成的zlib.lib存放为“..\Borland\CBuilder6\Lib\Zlib\zlib.lib”下。rnrn5、新建CB工程。在Project-Options-Directories/Conditions中分别将“..\CBuilder6\Include\Zlib”添加到Include Path中,以及将“..\Borland\CBuilder6\Lib\Zlib”添加到Library Path中。rnrn6、利用Project-Add to Project将“..\Borland\CBuilder6\Lib\Zlib\zlib.lib”文件添加到当前工程中。rnrn7、在unit1.cpp里添加#include "zlib.h",编辑运行,出错,rnrnE2015 Ambiguity between 'Byte' and 'System::Byte'rnrn修改方式:将该行的Byte改为System::Byte,保存再编辑通过。(因为是DLL的Zlib,貌似不能直接在CB里修改这个,利用txt文本打开修改保存,再在Cb里运行)rnrn8、添加一个button空间,输入代码,最后出问题,问题是:rnrn [Linker Error] Unresolved external '_deflateInit_' referenced from E:\C++BUILDER\SS\UNIT1.OBJrn [Linker Error] Unresolved external '_deflate' referenced from E:\C++BUILDER\SS\UNIT1.OBJrn [Linker Error] Unresolved external '_deflateEnd' referenced from E:\C++BUILDER\SS\UNIT1.OBJrnrn我不知道怎么修改了~~~郁闷。rnrn整个Unit1.cpp的源代码如下:rnrn//---------------------------------------------------------------------------rnrn#include rn#pragma hdrstoprnrn#include "Unit1.h"rn#include rn#include rn#include rn#include "zlib.h"rnrn#define CHUNK 16384rn//---------------------------------------------------------------------------rn#pragma package(smart_init)rn#pragma resource "*.dfm"rnTForm1 *Form1;rn//---------------------------------------------------------------------------rn__fastcall TForm1::TForm1(TComponent* Owner)rn : TForm(Owner)rnrnrn//---------------------------------------------------------------------------rn/* Compress from file source to file dest until EOF on source.rn def() returns Z_OK on success, Z_MEM_ERROR if memory could not bern allocated for processing, Z_STREAM_ERROR if an invalid compressionrn level is supplied, Z_VERSION_ERROR if the version of zlib.h and thern version of the library linked do not match, or Z_ERRNO if there isrn an error reading or writing the files. */rnint def(FILE *source, FILE *dest, int level)rnrn int ret, flush;rn unsigned have;rn z_stream strm;rn char in[CHUNK];rn char out[CHUNK];rnrn /* allocate deflate state */rn strm.zalloc = Z_NULL;rn strm.zfree = Z_NULL;rn strm.opaque = Z_NULL;rn ret = deflateInit(&strm, level);rn if (ret != Z_OK)rn return ret;rnrn /* compress until end of file */rn do rn strm.avail_in = fread(in, 1, CHUNK, source);rn if (ferror(source)) rn (void)deflateEnd(&strm);rn return Z_ERRNO;rn rn flush = feof(source) ? Z_FINISH : Z_NO_FLUSH;rn strm.next_in = in;rnrn /* run deflate() on input until output buffer not full, finishrn compression if all of source has been read in */rn do rn strm.avail_out = CHUNK;rn strm.next_out = out;rn ret = deflate(&strm, flush); /* no bad return value */rn assert(ret != Z_STREAM_ERROR); /* state not clobbered */rn have = CHUNK - strm.avail_out;rn if (fwrite(out, 1, have, dest) != have || ferror(dest)) rn (void)deflateEnd(&strm);rn return Z_ERRNO;rn rn while (strm.avail_out == 0);rn assert(strm.avail_in == 0); /* all input will be used */rnrn /* done when last data in file processed */rn while (flush != Z_FINISH);rn assert(ret == Z_STREAM_END); /* stream will be complete */rnrn /* clean up and return */rn (void)deflateEnd(&strm);rn return Z_OK;rnrn//----------------------rn/* Decompress from file source to file dest until stream ends or EOF.rn inf() returns Z_OK on success, Z_MEM_ERROR if memory could not bern allocated for processing, Z_DATA_ERROR if the deflate data isrn invalid or incomplete, Z_VERSION_ERROR if the version of zlib.h andrn the version of the library linked do not match, or Z_ERRNO if therern is an error reading or writing the files. */rnint inf(FILE *source, FILE *dest)rnrn int ret;rn unsigned have;rn z_stream strm;rn char in[CHUNK];rn char out[CHUNK];rnrn /* allocate inflate state */rn strm.zalloc = Z_NULL;rn strm.zfree = Z_NULL;rn strm.opaque = Z_NULL;rn strm.avail_in = 0;rn strm.next_in = Z_NULL;rn ret = inflateInit(&strm);rn if (ret != Z_OK)rn return ret;rnrn /* decompress until deflate stream ends or end of file */rn do rn strm.avail_in = fread(in, 1, CHUNK, source);rn if (ferror(source)) rn (void)inflateEnd(&strm);rn return Z_ERRNO;rn rn if (strm.avail_in == 0)rn break;rn strm.next_in = in;rnrn /* run inflate() on input until output buffer not full */rn do rn strm.avail_out = CHUNK;rn strm.next_out = out;rn ret = inflate(&strm, Z_NO_FLUSH);rn assert(ret != Z_STREAM_ERROR); /* state not clobbered */rn switch (ret) rn case Z_NEED_DICT:rn ret = Z_DATA_ERROR; /* and fall through */rn case Z_DATA_ERROR:rn case Z_MEM_ERROR:rn (void)inflateEnd(&strm);rn return ret;rn rn have = CHUNK - strm.avail_out;rn if (fwrite(out, 1, have, dest) != have || ferror(dest)) rn (void)inflateEnd(&strm);rn return Z_ERRNO;rn rn while (strm.avail_out == 0);rnrn /* done when inflate() says it's done */rn while (ret != Z_STREAM_END);rnrn /* clean up and return */rn (void)inflateEnd(&strm);rn return ret == Z_STREAM_END ? Z_OK : Z_DATA_ERROR;rnrnrn/* report a zlib or i/o error */rnvoid zerr(int ret)rnrn fputs("zpipe: ", stderr);rn switch (ret) rn case Z_ERRNO:rn if (ferror(stdin))rn fputs("error reading stdin\n", stderr);rn if (ferror(stdout))rn fputs("error writing stdout\n", stderr);rn break;rn case Z_STREAM_ERROR:rn fputs("invalid compression level\n", stderr);rn break;rn case Z_DATA_ERROR:rn fputs("invalid or incomplete deflate data\n", stderr);rn break;rn case Z_MEM_ERROR:rn fputs("out of memory\n", stderr);rn break;rn case Z_VERSION_ERROR:rn fputs("zlib version mismatch!\n", stderr);rn rnrn//-----------------------------------------rnrnvoid __fastcall TForm1::Button1Click(TObject *Sender)rnrn def((FILE *)"E:\\C++Builder\\ss\\zz.txt", (FILE *)"E:\\C++Builder\\ss\\out", Z_DEFAULT_COMPRESSION);rn// inf();rn// int ret;rn//rn// /* do compression if no arguments */rn// if (argc == 1) rn// ret = def(stdin, stdout, Z_DEFAULT_COMPRESSION);rn// if (ret != Z_OK)rn// zerr(ret);rn// return ret;rn// rn//rn// /* do decompression if -d specified */rn// else if (argc == 2 && strcmp(argv[1], "-d") == 0) rn// ret = inf(stdin, stdout);rn// if (ret != Z_OK)rn// zerr(ret);rn// return ret;rn// rn//rn// /* otherwise, report usage */rn// else rn// fputs("zpipe usage: zpipe [-d] < source > dest\n", stderr);rn// return 1;rn// rnrn//---------------------------------------------------------------------------rnrn这也是在Zlib网站上的源码里找的,但是我不知道如何修改了,请大家帮忙,谢谢。rnrn帖子较长,给各位添麻烦了~~~不好意思。 论坛

没有更多推荐了,返回首页