SRPC
Introduction
SRPC is an RPC system developed by Sogou. Its main features include:
- Base on Sogou C++ Workflow, with the following features:
- High performance
- Low development and access cost
- Compatible with SeriesWork and ParallelWork in Workflow
- One-click migration for existing projects with protobuf/thrift
- Support several IDL formats, including:
- Protobuf
- Thrift
- Support several data formats, including:
- Protobuf serialize
- Thrift Binary serialize
- JSON serialize
- Support several compression formats transparently, including:
- gzip
- zlib
- snappy
- lz4
- Support several communication protocols transparently, including:
- tcp
- http
- sctp
- ssl
- https
- With HTTP+JSON, you can use any language:
- As a server, you can accept POST requests with HTTP server developed in any language and parse the HTTP headers.
- As a client, you can send POST requests with HTTP client developed in any language and add the required HTTP headers.
- Built-in client/server which can seamlessly communicate with a server/client in other RPC frameworks, including:
- BRPC
- TRPC (the only open-source implementation of TRPC protocol so far)
GRPC- Thrift Framed Binary
- Thrift Http Binary
- How to use it together with Workflow:
- You can use the interface to create an RPC task
- You can put the RPC task into SeriesWork or ParallelWork, and you can also get the current SeriesWork in the callback.
- You can also use other features supported by Workflow, including upstream, calculation scheduling, asynchronous file IO, etc.
- More features and layers
Installation
- srpc is a static library, libsrpc.a. You only need to add the libsrpc as a dependency in the development environment, and it is not required in the compiled binary release.
- srpc depends on Workflow and protobuf3.
- For protobuf, you must install protobuf v3.0.0 or above by yourself.
- For Workflow, it`s added as dependency automatically via git submodule.
- For snappy and lz4, source codes are also included as third_party via git submodule.
git clone --recursive https://github.com/sogou/srpc.git
cd srpc
make
sudo make install
Tutorial
- Step 1: Design IDL description file
- Step 2: Implement ServiceIMPL
- Step 3: Start the Server
- Step 4: Use the Client
- Step 5: Understand asynchrous Context
- Step 6: Use it together with the Workflow
Easy to compile tutorial with these commands:
cd tutorial
make
Quick Start
1. example.proto
syntax = "proto3";// You can use either proto2 or proto3. Both are supported by srpc
message EchoRequest {
string message = 1;
string name = 2;
};
message EchoResponse {
string message = 1;
};
service Example {
rpc Echo(EchoRequest) returns (EchoResponse);
};
2. generate code
protoc example.proto --cpp_out=./ --proto_path=./
srpc_generator protobuf ./example.proto ./
3. server.cc
#include <stdio.h>
#include <signal.h>
#include "example.srpc.h"
using namespace srpc;
class ExampleServiceImpl : public Example::Service
{
public:
void Echo(EchoRequest *request, EchoResponse *response, RPCContext *ctx) override
{
response->set_message("Hi, " + request->name());
// gzip/zlib/snappy/lz4/none
// ctx->set_compress_type(RPCCompressGzip);
// protobuf/json
// ctx->set_data_type(RPCDataJson);
printf("get_req:\n%s\nset_resp:\n%s\n",
request->DebugString().c_str(), response->DebugString().c_str());
}
};
void sig_handler(int signo) { }
int main()
{
signal(SIGINT, sig_handler);
signal(SIGTERM, sig_handler);
SRPCServer server_tcp;
SRPCHttpServer server_http;
ExampleServiceImpl impl;
server_tcp.add_service(&impl);
server_http.add_service(&impl);
server_tcp.start(1412);
server_http.start(8811);
getchar(); // press "Enter" to end.
server_http.stop();
server_tcp.stop();
return 0;
}
4. client.cc
#include <stdio.h>
#include "example.srpc.h"
using namespace srpc;
int main()
{
Example::SRPCClient client("127.0.0.1", 1412);
EchoRequest req;
req.set_message("Hello, srpc!");
req.set_name("workflow");
client.Echo(&req, [](EchoResponse *response, RPCContext *ctx) {
if (ctx->success())
printf("%s\n", response->DebugString().c_str());
else
printf("status[%d] error[%d] errmsg:%s\n",
ctx->get_status_code(), ctx->get_error(), ctx->get_errmsg());
});
getchar(); // press "Enter" to end.
return 0;
}
5. make
These compile commands are only for Linux system. On other system, complete cmake in tutorial is recommanded.
g++ -o server server.cc example.pb.cc -std=c++11 -lsrpc
g++ -o client client.cc example.pb.cc -std=c++11 -lsrpc
6. run
Terminal 1
./server
Terminal 2
./client
curl 127.0.0.1:8811/Example/Echo -H 'Content-Type: application/json' -d '{message:"from curl",name:"CURL"}'
Output of Terminal 1
get_req:
message: "Hello, srpc!"
name: "workflow"
set_resp:
message: "Hi, workflow"
get_req:
message: "from curl"
name: "CURL"
set_resp:
message: "Hi, CURL"
Output of Terminal 2
message: "Hi, workflow"
{"message":"Hi, CURL"}
Benchmark
- CPU 2-chip/8-core/32-processor Intel(R) Xeon(R) CPU E5-2630 v3 @2.40GHz
- Memory all 128G
- 10 Gigabit Ethernet
- BAIDU brpc-client in pooled (connection pool) mode
QPS at cross-machine single client→ single server under different concurrency
Client = 1
ClientThread = 64, 128, 256, 512, 1024
RequestSize = 32
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16
QPS at cross-machine multi-client→ single server under different client processes
Client = 1, 2, 4, 8, 16
ClientThread = 32
RequestSize = 32
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16
QPS at same-machine single client→ single server under different concurrency
Client = 1
ClientThread = 1, 2, 4, 8, 16, 32, 64, 128, 256
RequestSize = 1024
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16
QPS at same-machine single client→ single server under different request sizes
Client = 1
ClientThread = 100
RequestSize = 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8192, 16384, 32768
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16
Latency CDF for fixed QPS at same-machine single client→ single server
Client = 1
ClientThread = 50
ClientQPS = 10000
RequestSize = 1024
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16
Outiler = 1%
Latency CDF for fixed QPS at cross-machine multi-client→ single server
Client = 32
ClientThread = 16
ClientQPS = 2500
RequestSize = 512
Duration = 20s
Server = 1
ServerIOThread = 16
ServerHandlerThread = 16
Outiler = 1%
Contact
- Email - [email protected] - main author
- Issue - You are very welcome to post questions to issues list.
- QQ - Group number:
618773193
搞了一天了，心累。
昨天没看issues，没用vcpkg，自己按照依赖用cmake一个个编译的，最终编译到tutorial的时候报错，运行时库不匹配，改好之后又报错重定义。最后实在没办法就按照issues里用vcpkg装好了依赖，然后一步步编译workflow、srpc，最后又到了tutorial，这次又出了新问题。
说实话，真的不想用了，看起来依赖不多，编译起来问题一堆，我现在就一个想法，赶紧的毁灭吧
