(三) Nepxion-Thunder分布式RPC集成框架 - 结构管理 -

nepxion

浏览: 37234 次
性别:
来自: 上海

最近访客更多访客>>

漂移361john

shenzlly

javac_xinyun

aipiciq

博主相关

博客

微博

相册

留言

关于我

文章分类

社区版块

存档分类

(三) Nepxion-Thunder分布式RPC集成框架 - 结构管理

博客分类：

开源框架
平台选型

Netty Hessian Kafka Redis Docker

Nepxion-Thunder（QQ 群 471164539）发布在https://github.com/Nepxion/

1. 类结构

1.1 cluster：负载均衡和远程注册中心集群服务地址同步

1.1.1 consistency - 远程服务上下线会产生事件，该模块线程安全维护本地服务列表和注册中心同步。该服务列表将用在负载均衡
1.1.2 loadbalance - 负载均衡。consistenthash为一致性Hash算法，random为随机轮循算法，roundrobin为权重轮循算法
1.2 common：通用类，实体类，工具类

1.2.1 callback - 异步回调接口实现
1.2.2 config - 持久化到注册中心的类，包括ApplicationConfig，ServiceConfig，ReferenceConfig
1.2.3 constants - 静态常量定义
1.2.4 container - 缓存和句柄容器定义，包括CacheContainer(线程安全的本地缓存容器，也同时利用它跟注册中心做同步)，ExecutorContainer(句柄和运行器容器，对于这两者，在Thunder的实现，对外都是通过接口暴露，而实现类是通过配置文件定义的方式进行反射装载，反射初始化后的对象都保存在ExecutorContainer里)
1.2.5 delegate - 委托类实现，主要是针对Properties，CacheContainer，ExecutorContainer的代理，在Spring扫描线程中将委托类注入到各个FactorBean
1.2.6 entity - 本地实体类
1.2.7 invocation - 用于反射调用
1.2.8 object - 对象池的实现
1.2.9 promise - 链式调用的实现
1.2.10 property - 读取本地和远程配置文件的实现
1.2.11 spi - Java SPI的实现
1.2.12 thread - 线程池的实现
1.2.13 time - 时间类的实现
1.2.14 util - 工具类的实现
1.3 event：异步事件发布

1.3.1 eventbus - 基于Google Guava的进程内发布/订阅机制
1.3.2 mobile - 基于EventBus的异常信息的短信通知，未实现
1.3.3 protocol - 基于EventBus的异常信息的发布/拦截，是其它事件通知模块的基础
1.3.4 registry - 基于EventBus的上下线通知
1.3.5 smtp - 基于EventBus的Smtp协议的邮件通知
1.4 framework：Spring扫描线程，初始化，包括对协议初始化，注册中心初始化等

1.4.1 bean - Spring FactoryBean
1.4.2 context - Spring上下文
1.4.3 exception - Spring相关异常
1.4.4 parser - Spring BeanDefinitionParser，用以解析FactoryBean
1.4.5 property - Spring属性解析
1.5 monitor：监控中心，包括日志监控(生成日志的方式)，缓存监控(把监控数据存到Redis缓存)，第三方WebService定义的监控(通过Http Post方式发送监控数据到Web服务器上)

1.5.1 MonitorExecutor.java - 把监控数据写往数据源的接口，旗下有1个抽象类和三个实现类
1.5.2 MonitorRetriever.java - 从数据源读取监控数据的接口，旗下有1个抽象类和两个实现类
1.6 protocol：协议，中间件的实现，包括Netty，Hessian，MQ，Redis等实现

1.6.1 apache - Apache同步和异步调用框架
1.6.2 hessian - Hessian同步和异步Http调用框架
1.6.3 kafka - Kafka消息队列调用通用框架
1.6.4 mq - ActiveMQ和Tibco消息队列调用通用框架
1.6.5 netty - Netty同步和异步Nio调用框架
1.6.6 redis - 作为Netty和Hessian发布/订阅调用框架
1.7 registry：注册中心的实现

1.7.1 RegistryExecutor.java - 注册中心的核心接口运行器
1.7.2 RegistryInitializer.java - 注册中心的初始化器
1.7.3 RegistryLauncher.javva - 外部系统访问注册中心的运行器
1.8 security：安全控制的实现，包括限流，密钥控制，版本控制
1.9 serialization：快速序列化对象的实现

1.9.1 binary - Java实体类和字节数组的序列化/反序列化
1.9.2 compression - 对binary模块运行结果的压缩/解压
1.9.3 json - Java实体类和Json字符串的转换

2. 包引用
1. Maven管理方式

Thunder默认基于Maven来管理
Thunder构建于JDK 1.7版本

Thunder引用了众多的第三方组件/开源框架，编译时候，需要如下的pom.xml文件做支持

<?xml version="1.0" encoding="UTF-8"?>
<project xmlns="http://maven.apache.org/POM/4.0.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
	xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
	<modelVersion>4.0.0</modelVersion>
	<groupId>nepxion</groupId>
	<artifactId>root</artifactId>
	<version>1.0.0</version>
	<packaging>pom</packaging>
	<name>Nepxion Root</name>
	<url>http://code.taobao.org/u/Nepxion/</url>
	<properties>
		<org.springframework.version>4.2.5.RELEASE</org.springframework.version>
		<org.apache.logging.log4j.version>2.5</org.apache.logging.log4j.version>
		<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
	</properties>
	<dependencies>
		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-lang3</artifactId>
			<version>3.4</version>
		</dependency>

		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-collections4</artifactId>
			<version>4.1</version>
		</dependency>

		<dependency>
			<groupId>commons-io</groupId>
			<artifactId>commons-io</artifactId>
			<version>2.4</version>
		</dependency>

		<dependency>
			<groupId>org.apache.commons</groupId>
			<artifactId>commons-pool2</artifactId>
			<version>2.4.2</version>
		</dependency>

		<dependency>
			<groupId>commons-configuration</groupId>
			<artifactId>commons-configuration</artifactId>
			<version>1.10</version>
		</dependency>

		<dependency>
			<groupId>commons-beanutils</groupId>
			<artifactId>commons-beanutils</artifactId>
			<version>1.9.2</version>
		</dependency>

		<dependency>
			<groupId>commons-jxpath</groupId>
			<artifactId>commons-jxpath</artifactId>
			<version>1.3</version>
		</dependency>

		<dependency>
			<groupId>commons-validator</groupId>
			<artifactId>commons-validator</artifactId>
			<version>1.5.0</version>
		</dependency>
		
		<dependency>
			<groupId>com.sun.mail</groupId>
			<artifactId>javax.mail</artifactId>
			<version>1.5.5</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-core</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-beans</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-aop</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-context</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-context-support</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-web</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-webmvc</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-jms</artifactId>
			<version>${org.springframework.version}</version>
		</dependency>

		<dependency>
			<groupId>org.springframework</groupId>
			<artifactId>spring-test</artifactId>
			<version>${org.springframework.version}</version>
			<scope>test</scope>
		</dependency>
		
		<dependency>
			<groupId>io.netty</groupId>
			<artifactId>netty-all</artifactId>
			<version>4.0.36.Final</version>
		</dependency>

		<dependency>
			<groupId>com.caucho</groupId>
			<artifactId>hessian</artifactId>
			<version>4.0.38</version>
		</dependency>

		<dependency>
			<groupId>org.apache.activemq</groupId>
			<artifactId>activemq-core</artifactId>
			<version>5.7.0</version>
			<exclusions>
				<exclusion>
					<groupId>org.slf4j</groupId>
					<artifactId>slf4j-api</artifactId>
				</exclusion>
			</exclusions>
		</dependency>

		<dependency>
			<groupId>org.apache.activemq</groupId>
			<artifactId>activemq-pool</artifactId>
			<version>5.13.2</version>
		</dependency>

		<dependency>
			<groupId>org.apache.kafka</groupId>
			<artifactId>kafka-clients</artifactId>
			<version>0.9.0.1</version>
		</dependency>
		
		<dependency>
			<groupId>redis.clients</groupId>
			<artifactId>jedis</artifactId>
			<version>2.8.1</version>
		</dependency>

		<!-- Curator Framework 2.9.1 or up won't be compatible -->
		<dependency>
			<groupId>org.apache.curator</groupId>
			<artifactId>curator-framework</artifactId>
			<version>2.9.0</version>
			<exclusions>
				<exclusion>
					<groupId>jline</groupId>
					<artifactId>jline</artifactId>
				</exclusion>
				<exclusion>
					<groupId>log4j</groupId>
					<artifactId>log4j</artifactId>
				</exclusion>
				<exclusion>
					<groupId>io.netty</groupId>
					<artifactId>netty</artifactId>
				</exclusion>
				<exclusion>
					<groupId>com.google.guava</groupId>
					<artifactId>guava</artifactId>
				</exclusion>
			</exclusions>
		</dependency>

		<dependency>
			<groupId>org.apache.curator</groupId>
			<artifactId>curator-recipes</artifactId>
			<version>2.9.0</version>
			<exclusions>
				<exclusion>
					<groupId>jline</groupId>
					<artifactId>jline</artifactId>
				</exclusion>
				<exclusion>
					<groupId>log4j</groupId>
					<artifactId>log4j</artifactId>
				</exclusion>
				<exclusion>
					<groupId>io.netty</groupId>
					<artifactId>netty</artifactId>
				</exclusion>
				<exclusion>
					<groupId>com.google.guava</groupId>
					<artifactId>guava</artifactId>
				</exclusion>
			</exclusions>
		</dependency>

		<dependency>
			<groupId>de.ruedigermoeller</groupId>
			<artifactId>fst</artifactId>
			<version>2.45</version>
			<exclusions>
				<exclusion>
					<groupId>org.javassist</groupId>
					<artifactId>javassist</artifactId>
				</exclusion>
			</exclusions>
		</dependency>
		
		<dependency>
			<groupId>com.esotericsoftware</groupId>
			<artifactId>kryo</artifactId>
			<version>3.0.3</version>
		</dependency>
		
		<dependency>
			<groupId>io.protostuff</groupId>
			<artifactId>protostuff-core</artifactId>
			<version>1.4.0</version>
		</dependency>
		
		<dependency>
			<groupId>io.protostuff</groupId>
			<artifactId>protostuff-runtime</artifactId>
			<version>1.4.0</version>
		</dependency>
		
		<dependency>
			<groupId>com.facebook.jcommon</groupId>
			<artifactId>util</artifactId>
			<version>0.1.20</version>
		</dependency>

		<dependency>
			<groupId>aopalliance</groupId>
			<artifactId>aopalliance</artifactId>
			<version>1.0</version>
		</dependency>

		<dependency>
			<groupId>com.google.guava</groupId>
			<artifactId>guava</artifactId>
			<version>19.0</version>
		</dependency>
		
		<dependency>
			<groupId>org.jdeferred</groupId>
			<artifactId>jdeferred-core</artifactId>
			<version>1.2.4</version>
		</dependency>

		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-core</artifactId>
			<version>2.7.3</version>
		</dependency>

		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-databind</artifactId>
			<version>2.7.3</version>
		</dependency>

		<dependency>
			<groupId>com.fasterxml.jackson.core</groupId>
			<artifactId>jackson-annotations</artifactId>
			<version>2.7.3</version>
		</dependency>
		
		<dependency>
			<groupId>com.alibaba</groupId>
			<artifactId>fastjson</artifactId>
			<version>1.2.8</version>
		</dependency>

		<dependency>
			<groupId>com.lmax</groupId>
			<artifactId>disruptor</artifactId>
			<version>3.3.4</version>
		</dependency>

		<dependency>
			<groupId>org.apache.httpcomponents</groupId>
			<artifactId>httpclient</artifactId>
			<version>4.5.2</version>
		</dependency>

		<dependency>
			<groupId>org.apache.httpcomponents</groupId>
			<artifactId>httpasyncclient</artifactId>
			<version>4.1.1</version>
		</dependency>

		<dependency>
			<groupId>javax.servlet</groupId>
			<artifactId>javax.servlet-api</artifactId>
			<version>3.1.0</version>
		</dependency>

		<dependency>
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-core</artifactId>
			<version>${org.apache.logging.log4j.version}</version>
		</dependency>

		<dependency>
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-1.2-api</artifactId>
			<version>${org.apache.logging.log4j.version}</version>
		</dependency>

		<dependency>
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-slf4j-impl</artifactId>
			<version>${org.apache.logging.log4j.version}</version>
		</dependency>

		<dependency>
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-jcl</artifactId>
			<version>${org.apache.logging.log4j.version}</version>
		</dependency>

		<dependency>
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-jul</artifactId>
			<version>${org.apache.logging.log4j.version}</version>
		</dependency>

		<dependency>
			<groupId>org.apache.logging.log4j</groupId>
			<artifactId>log4j-web</artifactId>
			<version>${org.apache.logging.log4j.version}</version>
		</dependency>

		<dependency>
			<groupId>junit</groupId>
			<artifactId>junit</artifactId>
			<version>4.12</version>
		</dependency>
	</dependencies>
</project>

Thunder在第三方运行的时候，只需要引入确定的通信中间件即可。例如采用Netty通信，只需要引入Netty相关的包。当然，一些必要的基础包必须引入，它们包括：
Apache Common包
Spring Framework包
Curator包 (支撑Zookeeper框架)
FST包
Fast Json包
Log4J2包 (异步)
Redis包 (当用Netty和Hessian做发布/订阅功能，以及日志入库的时候，需要用到它)
Apache HttpComponents (同步/异步包，当用第三方WebService做监控的时候，需要用到它)
Google Guava包(做EventBus用)
其它基础框架进入的包

2. IVY管理方式

如果你想通过IVY来管理Thunder，参考下面的格式

<?xml version="1.0" encoding="UTF-8"?>
<ivy-module version="1.0">
	<info organisation="your_package" module="your_module"/>
	<configurations>
		<conf name="default"/>
		<conf name="compile"/>
		<conf name="zip"/>
		<conf name="test"/>
	</configurations>
	<publications>
		<artifact ext="jar"/>
	</publications>
	<dependencies>
	    <!-- 缩进代表上面包的关联引用包-->
	    <!-- Apache Common，必需 -->
	    <dependency org="org.apache" name="commons-lang3" rev="3.4" conf="zip->default"/>
		<dependency org="org.apache" name="commons-collections4" rev="4.1" conf="zip->default"/>
		<dependency org="commons-io" name="commons-io" rev="2.4" conf="zip->default"/>
		<dependency org="org.apache" name="commons-pool2" rev="2.4.2" conf="zip->default"/>
		<dependency org="org.apache" name="commons-configuration" rev="1.10" conf="zip->default"/>
			<dependency org="org.apache" name="commons-lang" rev="2.6" conf="zip->default"/>
			<dependency org="org.apache" name="commons-logging" rev="1.1.1" conf="zip->default"/>
		<dependency org="commons-beanutils" name="commons-beanutils" rev="1.9.2" conf="zip->default"/>
			<dependency org="org.apache" name="commons-collections" rev="3.2.1" conf="zip->default"/>
		<dependency org="org.apache" name="commons-jxpath" rev="1.3" conf="zip->default"/>
		<dependency org="org.apache" name="commons-validator" rev="1.5.0" conf="zip->default"/>
			<dependency org="org.apache" name="commons-digester" rev="1.8.1" conf="zip->default"/>
		
		<!-- Spring，必需，但也可以用低版本代替 -->
		<dependency org="org.springframework" name="spring-core" rev="4.2.4.RELEASE" conf="zip->default"/>
		<dependency org="org.springframework" name="spring-beans" rev="4.2.4.RELEASE" conf="zip->default"/>
		<dependency org="org.springframework" name="spring-aop" rev="4.2.4.RELEASE" conf="zip->default"/>
		<dependency org="org.springframework" name="spring-context" rev="4.2.4.RELEASE" conf="zip->default"/>
			<dependency org="org.springframework" name="spring-expression" rev="4.2.4.RELEASE" conf="zip->default"/>		
		<dependency org="org.springframework" name="spring-context-support" rev="4.2.4.RELEASE" conf="zip->default"/>
		<dependency org="org.springframework" name="spring-web" rev="4.2.4.RELEASE" conf="zip->default"/>
		<dependency org="org.springframework" name="spring-webmvc" rev="4.2.4.RELEASE" conf="zip->default"/>
		<dependency org="org.springframework" name="spring-jms" rev="4.2.4.RELEASE" conf="zip->default"/>
			<dependency org="org.springframework" name="spring-messaging" rev="4.2.4.RELEASE" conf="zip->default"/>
			<dependency org="org.springframework" name="spring-tx" rev="4.2.4.RELEASE" conf="zip->default"/>
		<dependency org="org.springframework" name="spring-test" rev="4.2.4.RELEASE" conf="test->default"/>
				
		<!-- Zookeeper，必需 -->
		<dependency org="org.apache.curator" name="curator-framework" rev="2.9.0" conf="zip->default"/>
			<dependency org="org.apache.curator" name="curator-client" rev="2.9.0" conf="zip->default"/>
				<dependency org="org.apache" name="zookeeper" rev="3.4.6" conf="zip->default"/>		
		<dependency org="org.apache.curator" name="curator-recipes" rev="2.9.0" conf="zip->default"/>
		
		<!-- Netty，必需，与Kafka二选一 -->
		<dependency org="io.netty" name="netty-all" rev="4.0.34.Final" conf="zip->default"/>
		
		<!-- Kafka，必需，与Kafka二选一 -->
		<dependency org="org.apache.kafka" name="kafka-clients" rev="0.9.0.1" conf="zip->default"/>
			<dependency org="org.slf4j" name="slf4j-api" rev="1.7.6" conf="zip->default"/>
			<dependency org="org.xerial" name="snappy-java" rev="1.1.1.7" conf="zip->default"/>
			<dependency org="net.jpountz.lz4" name="lz4" rev="1.2.0" conf="zip->default"/>
		
		<!-- Redis，必需 -->
		<dependency org="redis.clients" name="jedis" rev="2.8.0" conf="zip->default"/>
		
		<!-- FST的二进制序列化，必需 -->
		<dependency org="de.ruedigermoeller" name="fst" rev="2.45" conf="zip->default"/>
			<dependency org="org.objenesis" name="objenesis" rev="2.1" conf="zip->default"/>
		
		<!-- Json的Jaskson序列化，必需 -->
		<dependency org="com.fasterxml.jackson.core" name="jackson-core" rev="2.7.2" conf="zip->default"/>
		<dependency org="com.fasterxml.jackson.core" name="jackson-databind" rev="2.7.2" conf="zip->default"/>
		<dependency org="com.fasterxml.jackson.core" name="jackson-annotations" rev="2.7.2" conf="zip->default"/>

		<!-- Google Guava，必需 -->		
		<dependency org="com.google" name="guava" rev="19.0" conf="zip->default"/>
		
		<!-- Log4J2，必需，但也可以用其它方式代替 -->
		<dependency org="org.apache.logging.log4j" name="log4j-core" rev="2.3" conf="test->default"/>
			<dependency org="org.apache.logging.log4j" name="log4j-api" rev="2.3" conf="test->default"/>
		<dependency org="org.apache.logging.log4j" name="log4j-1.2-api" rev="2.3" conf="test->default"/>
		<dependency org="org.apache.logging.log4j" name="log4j-slf4j-impl" rev="2.3" conf="test->default"/>
		<dependency org="org.apache.logging.log4j" name="log4j-jcl" rev="2.3" conf="test->default"/>
		<dependency org="org.apache.logging.log4j" name="log4j-jul" rev="2.3" conf="test->default"/>
		<dependency org="org.apache.logging.log4j" name="log4j-web" rev="2.3" conf="test->default"/>
		
		<!-- Aop，必需 -->
		<dependency org="unknown" name="aopalliance" rev="1.0" conf="zip->default"/>
		
		
		<!-- Promise链式调用框架，非必需 -->		
		<dependency org="org.jdeferred" name="jdeferred-core" rev="1.2.4" conf="zip->default"/>
		
		<!-- Kryo的二进制序列化，非必需 -->
		<dependency org="com.esotericsoftware" name="kryo" rev="3.0.3" conf="zip->default"/>
			<dependency org="com.esotericsoftware" name="reflectasm" rev="1.10.1" conf="zip->default"/>
				<dependency org="org.objectweb" name="asm" rev="5.0.3" conf="zip->default"/>
			<dependency org="com.esotericsoftware" name="minlog" rev="1.3.0" conf="zip->default"/>
			
		<!-- Protostuff的二进制序列化，非必需 -->	
		<dependency org="io.protostuff" name="protostuff-core" rev="1.4.0" conf="zip->default"/>
			<dependency org="io.protostuff" name="protostuff-api" rev="1.4.0" conf="zip->default"/>	
		<dependency org="io.protostuff" name="protostuff-runtime" rev="1.4.0" conf="zip->default"/>
			<dependency org="io.protostuff" name="protostuff-collectionschema" rev="1.4.0" conf="zip->default"/>
		
		<!-- Alibaba的Jaskson序列化，非必需 -->
		<dependency org="com.alibaba" name="fastjson" rev="1.2.8" conf="zip->default"/>
		
		<!-- Facebook的QuickLz压缩算法，非必需 -->
		<dependency org="com.facebook.jcommon" name="util" rev="0.1.20" conf="zip->default"/>
		
		<!-- Disruptor的异步事件机制，非必需 -->
		<dependency org="com.lmax" name="disruptor" rev="3.3.4" conf="zip->default"/>
		
		<!-- Apache Http，非必需 -->
		<dependency org="org.apache.httpcomponents" name="httpclient" rev="4.5.2" conf="zip->default"/>
			<dependency org="org.apache.httpcomponents" name="httpcore" rev="4.4.4" conf="zip->default"/>
			<dependency org="commons-codec" name="commons-codec" rev="1.9" conf="zip->default"/>
		<dependency org="org.apache.httpcomponents" name="httpasyncclient" rev="4.1.1" conf="zip->default"/>
			<dependency org="org.apache.httpcomponents" name="httpcore-nio" rev="4.4.4" conf="zip->default"/>
				
		<!-- Mail，如果要用到邮件通知功能需要引用该包，非必需 -->
		<dependency org="com.sun.mail" name="javax.mail" rev="1.5.5" conf="zip->default"/>
			<dependency org="com.sun" name="activation" rev="1.1" conf="zip->default"/>
		
		<!-- Splunk，如果要用到Splunk相关API需要引用该包，非必需 -->
		<dependency org="com.splunk" name="splunk-sdk-java" rev="1.5.0" conf="zip->default"/>
	
		<!-- 单元测试，非必需 -->
		<dependency org="junit" name="junit-dep" rev="4.11" conf="test->default"/>
			<dependency org="org.hamcrest" name="hamcrest-core" rev="1.3" conf="test->default"/>
	</dependencies>
</ivy-module>