<rt id="bn8ez"></rt>
<label id="bn8ez"></label>

  • <span id="bn8ez"></span>

    <label id="bn8ez"><meter id="bn8ez"></meter></label>

    from:http://pfmiles.github.io/blog/recently-hessian-deserialize-problem-and-thread-pool-executor-experience/

    最近工作中遇到一個(gè)詭異的問(wèn)題:別人遠(yuǎn)程調(diào)用我們的系統(tǒng)暴露的服務(wù),同步調(diào)用,底層使用hessian協(xié)議做序列化;
    調(diào)用方系統(tǒng)報(bào)空指針,反序列化失敗:

    2013-04-18 16:52:10,308 [AvatarRuleChargeService.java:74] [com.alibaba.itbu.billing.biz.adaptor.avatar.AvatarRuleChargeService] ERROR com.alibaba.itbu.billing.biz.adaptor.crm.ChargeProxy :: avatar charge sys error com.alibaba.dubbo.rpc.RpcException: Failed to invoke the method match in the service com.alibaba.china.ruleservice.RuleService. Tried 3 times of the providers [172.22.6.83:20980, 172.22.6.80:20980, 172.22.9.76:20980] (3/3) from the registry dubbo-reg1.hst.xyi.cn.alidc.net:9090 on the consumer 172.30.118.26 using the dubbo version 2.4.9. Last error is: Failed to invoke remote method: match, provider: dubbo://172.22.6.83:20980/com.alibaba.china.ruleservice.RuleService?anyhost=true&application=billing&check=false&default.reference.filter=dragoon&dubbo=2.4.9&interface=com.alibaba.china.ruleservice.RuleService&methods=match&pid=18616&revision=1.0-SNAPSHOT&side=consumer&timeout=5000&timestamp=1366275108588&version=1.0.0, cause: com.alibaba.com.caucho.hessian.io.HessianProtocolException: 'com.alibaba.china.ruleservice.RuleServiceImpl$DynamicPluginInvocationMatchedResult' could not be instantiated com.alibaba.com.caucho.hessian.io.HessianProtocolException: 'com.alibaba.china.ruleservice.RuleServiceImpl$DynamicPluginInvocationMatchedResult' could not be instantiated     at com.alibaba.com.caucho.hessian.io.JavaDeserializer.instantiate(JavaDeserializer.java:275)     at com.alibaba.com.caucho.hessian.io.JavaDeserializer.readObject(JavaDeserializer.java:155)     at com.alibaba.com.caucho.hessian.io.SerializerFactory.readObject(SerializerFactory.java:396)     at com.alibaba.com.caucho.hessian.io.Hessian2Input.readObjectInstance(Hessian2Input.java:2070)     at com.alibaba.com.caucho.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2005)     at com.alibaba.com.caucho.hessian.io.Hessian2Input.readObject(Hessian2Input.java:1990)     at com.alibaba.com.caucho.hessian.io.Hessian2Input.readObject(Hessian2Input.java:1538)     at com.alibaba.dubbo.common.serialize.support.hessian.Hessian2ObjectInput.readObject(Hessian2ObjectInput.java:94)     at com.alibaba.dubbo.common.serialize.support.hessian.Hessian2ObjectInput.readObject(Hessian2ObjectInput.java:99)     at com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcResult.decode(DecodeableRpcResult.java:83)     at com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcResult.decode(DecodeableRpcResult.java:109)     at com.alibaba.dubbo.rpc.protocol.dubbo.DubboCodec.decodeBody(DubboCodec.java:97)     at com.alibaba.dubbo.remoting.exchange.codec.ExchangeCodec.decode(ExchangeCodec.java:128)     at com.alibaba.dubbo.remoting.exchange.codec.ExchangeCodec.decode(ExchangeCodec.java:87)     at com.alibaba.dubbo.rpc.protocol.dubbo.DubboCountCodec.decode(DubboCountCodec.java:49)     at com.alibaba.dubbo.remoting.transport.netty.NettyCodecAdapter$InternalDecoder.messageReceived(NettyCodecAdapter.java:135)     at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)     at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)     at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)     at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)     at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)     at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:349)     at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:280)     at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:200)     at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)     at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)     at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.reflect.InvocationTargetException     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)     at java.lang.reflect.Constructor.newInstance(Constructor.java:513)     at com.alibaba.com.caucho.hessian.io.JavaDeserializer.instantiate(JavaDeserializer.java:271)     ... 28 more Caused by: java.lang.NullPointerException     at com.alibaba.china.ruleservice.RuleServiceImpl$DynamicPluginInvocationMatchedResult.<init>(RuleServiceImpl.java:163)     ... 33 more      at com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:101)     at com.alibaba.dubbo.rpc.cluster.support.AbstractClusterInvoker.invoke(AbstractClusterInvoker.java:226)     at com.alibaba.dubbo.rpc.cluster.support.wrapper.MockClusterInvoker.invoke(MockClusterInvoker.java:72)     at com.alibaba.dubbo.rpc.proxy.InvokerInvocationHandler.invoke(InvokerInvocationHandler.java:52)     at com.alibaba.dubbo.common.bytecode.proxy1.match(proxy1.java)     at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)     at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)     at java.lang.reflect.Method.invoke(Method.java:597)     at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:307)     at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:182)     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:149)     at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:88)     at com.alibaba.itbu.billing.framework.aop.OpenApiLogAspect.logExecuteTime(OpenApiLogAspect.java:38)     at sun.reflect.GeneratedMethodAccessor199.invoke(Unknown Source)     at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)     at java.lang.reflect.Method.invoke(Method.java:597)     at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)     at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)     at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:64)     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)     at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:89)     at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:171)     at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)     at $Proxy107.match(Unknown Source)     at com.alibaba.itbu.billing.biz.adaptor.avatar.AvatarRuleChargeService.chargeByFactor(AvatarRuleChargeService.java:72)     at com.alibaba.itbu.billing.biz.charge.times.RuleChargeByTimesProcessor.getChargeResult(RuleChargeByTimesProcessor.java:62)     at com.alibaba.itbu.billing.biz.charge.times.ChargeByTimesProcessor.charge(ChargeByTimesProcessor.java:117)     at com.alibaba.itbu.billing.biz.task.ChargeByIncInstantTimesTask.charge(ChargeByIncInstantTimesTask.java:174)     at com.alibaba.itbu.billing.biz.task.ChargeByIncInstantTimesTask$1.run(ChargeByIncInstantTimesTask.java:109)     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)     at java.lang.Thread.run(Thread.java:662) Caused by: com.alibaba.dubbo.remoting.RemotingException: com.alibaba.com.caucho.hessian.io.HessianProtocolException: 'com.alibaba.china.ruleservice.RuleServiceImpl$DynamicPluginInvocationMatchedResult' could not be instantiated com.alibaba.com.caucho.hessian.io.HessianProtocolException: 'com.alibaba.china.ruleservice.RuleServiceImpl$DynamicPluginInvocationMatchedResult' could not be instantiated     at com.alibaba.com.caucho.hessian.io.JavaDeserializer.instantiate(JavaDeserializer.java:275)     at com.alibaba.com.caucho.hessian.io.JavaDeserializer.readObject(JavaDeserializer.java:155)     at com.alibaba.com.caucho.hessian.io.SerializerFactory.readObject(SerializerFactory.java:396)     at com.alibaba.com.caucho.hessian.io.Hessian2Input.readObjectInstance(Hessian2Input.java:2070)     at com.alibaba.com.caucho.hessian.io.Hessian2Input.readObject(Hessian2Input.java:2005)     at com.alibaba.com.caucho.hessian.io.Hessian2Input.readObject(Hessian2Input.java:1990)     at com.alibaba.com.caucho.hessian.io.Hessian2Input.readObject(Hessian2Input.java:1538)     at com.alibaba.dubbo.common.serialize.support.hessian.Hessian2ObjectInput.readObject(Hessian2ObjectInput.java:94)     at com.alibaba.dubbo.common.serialize.support.hessian.Hessian2ObjectInput.readObject(Hessian2ObjectInput.java:99)     at com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcResult.decode(DecodeableRpcResult.java:83)     at com.alibaba.dubbo.rpc.protocol.dubbo.DecodeableRpcResult.decode(DecodeableRpcResult.java:109)     at com.alibaba.dubbo.rpc.protocol.dubbo.DubboCodec.decodeBody(DubboCodec.java:97)     at com.alibaba.dubbo.remoting.exchange.codec.ExchangeCodec.decode(ExchangeCodec.java:128)     at com.alibaba.dubbo.remoting.exchange.codec.ExchangeCodec.decode(ExchangeCodec.java:87)     at com.alibaba.dubbo.rpc.protocol.dubbo.DubboCountCodec.decode(DubboCountCodec.java:49)     at com.alibaba.dubbo.remoting.transport.netty.NettyCodecAdapter$InternalDecoder.messageReceived(NettyCodecAdapter.java:135)     at org.jboss.netty.channel.SimpleChannelUpstreamHandler.handleUpstream(SimpleChannelUpstreamHandler.java:80)     at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:564)     at org.jboss.netty.channel.DefaultChannelPipeline.sendUpstream(DefaultChannelPipeline.java:559)     at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:274)     at org.jboss.netty.channel.Channels.fireMessageReceived(Channels.java:261)     at org.jboss.netty.channel.socket.nio.NioWorker.read(NioWorker.java:349)     at org.jboss.netty.channel.socket.nio.NioWorker.processSelectedKeys(NioWorker.java:280)     at org.jboss.netty.channel.socket.nio.NioWorker.run(NioWorker.java:200)     at org.jboss.netty.util.ThreadRenamingRunnable.run(ThreadRenamingRunnable.java:108)     at org.jboss.netty.util.internal.DeadLockProofWorker$1.run(DeadLockProofWorker.java:44)     at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)     at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)     at java.lang.Thread.run(Thread.java:662) Caused by: java.lang.reflect.InvocationTargetException     at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)     at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)     at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)     at java.lang.reflect.Constructor.newInstance(Constructor.java:513)     at com.alibaba.com.caucho.hessian.io.JavaDeserializer.instantiate(JavaDeserializer.java:271)     ... 28 more Caused by: java.lang.NullPointerException     at com.alibaba.china.ruleservice.RuleServiceImpl$DynamicPluginInvocationMatchedResult.<init>(RuleServiceImpl.java:163)     ... 33 more      at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.returnFromResponse(DefaultFuture.java:190)     at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:110)     at com.alibaba.dubbo.remoting.exchange.support.DefaultFuture.get(DefaultFuture.java:84)     at com.alibaba.dubbo.rpc.protocol.dubbo.DubboInvoker.doInvoke(DubboInvoker.java:96)     at com.alibaba.dubbo.rpc.protocol.AbstractInvoker.invoke(AbstractInvoker.java:144)     at com.alibaba.dubbo.rpc.listener.ListenerInvokerWrapper.invoke(ListenerInvokerWrapper.java:74)     at com.alibaba.dubbo.monitor.dragoon.filter.DragoonFilter.invoke0(DragoonFilter.java:82)     at com.alibaba.dubbo.monitor.dragoon.filter.DragoonFilter.invoke(DragoonFilter.java:34)     at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91)     at com.alibaba.dubbo.monitor.support.MonitorFilter.invoke(MonitorFilter.java:75)     at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91)     at com.alibaba.dubbo.rpc.protocol.dubbo.filter.FutureFilter.invoke(FutureFilter.java:53)     at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91)     at com.alibaba.dubbo.rpc.filter.ConsumerContextFilter.invoke(ConsumerContextFilter.java:48)     at com.alibaba.dubbo.rpc.protocol.ProtocolFilterWrapper$1.invoke(ProtocolFilterWrapper.java:91)     at com.alibaba.dubbo.rpc.protocol.InvokerWrapper.invoke(InvokerWrapper.java:53)     at com.alibaba.dubbo.rpc.cluster.support.FailoverClusterInvoker.doInvoke(FailoverClusterInvoker.java:77)     ... 32 more 

    看到這個(gè)日志第一反映是覺(jué)得RuleServiceImpl.java:163數(shù)據(jù)錯(cuò)誤,拋空指針,但檢查那塊代碼發(fā)現(xiàn)那個(gè)地方根本不可能拋空指針 —— 所有用到的變量都是new出來(lái)的;
    而且,依據(jù)調(diào)用方提供的錯(cuò)誤日志的拋出時(shí)間,我在被調(diào)用方系統(tǒng)的所有機(jī)器的日志里查找了一遍,沒(méi)有發(fā)現(xiàn)對(duì)應(yīng)的服務(wù)端日志;照理說(shuō)服務(wù)端拋空指針,服務(wù)端也會(huì)有對(duì)應(yīng)日志,但沒(méi)有任何線索…

    因此只好翻開(kāi)了JavaDeserializer.java:275作檢查,發(fā)現(xiàn)有這么一段:

      public JavaDeserializer(Class cl)   {     _type = cl;     _fieldMap = getFieldMap(cl);      _readResolve = getReadResolve(cl);      if (_readResolve != null) {       _readResolve.setAccessible(true);     }      Constructor []constructors = cl.getDeclaredConstructors();     long bestCost = Long.MAX_VALUE;      for (int i = 0; i < constructors.length; i++) {       Class []param = constructors[i].getParameterTypes();       long cost = 0;        for (int j = 0; j < param.length; j++) {     cost = 4 * cost;      if (Object.class.equals(param[j]))       cost += 1;     else if (String.class.equals(param[j]))       cost += 2;     else if (int.class.equals(param[j]))       cost += 3;     else if (long.class.equals(param[j]))       cost += 4;     else if (param[j].isPrimitive())       cost += 5;     else       cost += 6;       }        if (cost < 0 || cost > (1 << 48))     cost = 1 << 48;        cost += param.length << 48;        if (cost < bestCost) {         _constructor = constructors[i];         bestCost = cost;       }     }      if (_constructor != null) {       _constructor.setAccessible(true);       Class []params = _constructor.getParameterTypes();       _constructorArgs = new Object[params.length];       for (int i = 0; i < params.length; i++) {         _constructorArgs[i] = getParamArg(params[i]);       }     }   } 

    看完這段后,再結(jié)合遠(yuǎn)程調(diào)用的返回結(jié)果類(lèi)后恍然大悟:
    上面這段代碼,是hessian在反序列化的時(shí)候,用于在被反序列化的類(lèi)里面找一個(gè)“得分最低”的構(gòu)造函數(shù),反序列化時(shí)會(huì)加以調(diào)用;
    構(gòu)造函數(shù)的“得分”規(guī)則大致是:參數(shù)越少得分越低;參數(shù)個(gè)數(shù)相同時(shí),參數(shù)類(lèi)型越接近JDK內(nèi)置類(lèi)得分越低
    而我們的調(diào)用返回的類(lèi)只有一個(gè)構(gòu)造函數(shù),當(dāng)然只有這個(gè)構(gòu)造函數(shù)會(huì)被選中
    但是,hessian反序列化調(diào)用被選中的構(gòu)造函數(shù)時(shí),是這樣來(lái)創(chuàng)造該構(gòu)造函數(shù)需要的參數(shù)的:

    protected static Object getParamArg(Class cl)   {     if (! cl.isPrimitive())       return null;     else if (boolean.class.equals(cl))       return Boolean.FALSE;     else if (byte.class.equals(cl))       return new Byte((byte) 0);     else if (short.class.equals(cl))       return new Short((short) 0);     else if (char.class.equals(cl))       return new Character((char) 0);     else if (int.class.equals(cl))       return new Integer(0);     else if (long.class.equals(cl))       return new Long(0);     else if (float.class.equals(cl))       return new Float(0);     else if (double.class.equals(cl))       return new Double(0);     else       throw new UnsupportedOperationException();   } 

    可以看到,如果參數(shù)不是primitive類(lèi)型,會(huì)被null代替;但不巧的是我們的構(gòu)造函數(shù)內(nèi)部對(duì)該參數(shù)調(diào)用了一個(gè)方法,因此拋出了空指針…
    也就是說(shuō)這個(gè)空指針是拋在客戶(hù)端反序列化的時(shí)候而不是服務(wù)端內(nèi)部,因此服務(wù)端當(dāng)然找不到對(duì)應(yīng)的錯(cuò)誤日志了;
    解決的辦法也很簡(jiǎn)單:可以新增一個(gè)無(wú)參構(gòu)造函數(shù)(無(wú)參構(gòu)造函數(shù)肯定“得分”最低一定會(huì)被選中);或者修改代碼保證任意參數(shù)為null的時(shí)候都不會(huì)出問(wèn)題


    下面這個(gè)問(wèn)題更有意思,說(shuō)的是ThreadPoolExecutorRejectedExecutionHandler的使用:

    threadPool = new ThreadPoolExecutor(5, maxThreadNum, 5, TimeUnit.MINUTES, new SynchronousQueue<Runnable>(), tf,             new RejectedExecutionHandler() {                 public void rejectedExecution(Runnable r, ThreadPoolExecutor e) {                     logger.error("Ep thread pool exhausted, epId: " + id + "!!");                     r.run();                 }             }); 

    這個(gè)代碼是說(shuō),當(dāng)我這個(gè)ThreadPool不夠用,又不能再新增線程數(shù)的時(shí)候,由調(diào)用方線程自己來(lái)執(zhí)行這個(gè)Runnable任務(wù)…
    本來(lái)這看上去沒(méi)什么問(wèn)題,問(wèn)題出在這個(gè)Runnable本身的實(shí)現(xiàn)上 —— 它內(nèi)部將執(zhí)行它的線程block到了一個(gè)blocking queue上面,當(dāng)調(diào)用方主線程親自來(lái)執(zhí)行它時(shí),使得主線程再也回不去做它自己該做的事情了,因此會(huì)出大問(wèn)題…
    所以這么看來(lái),“當(dāng)線程池不夠用就讓調(diào)用方線程自己來(lái)干”的這個(gè)策略在實(shí)際使用時(shí)要非常謹(jǐn)慎

    這個(gè)問(wèn)題是通過(guò)一個(gè)方便的thread dump分析工具: tda來(lái)查找的,因?yàn)槌鰡?wèn)題的這個(gè)應(yīng)用是個(gè)多線程程序,有1600多個(gè)常駐線程,thread dump非常之大,肉眼直接看很不方便,但tda能將thread dump變得更友好易讀,方便排查問(wèn)題,在此推薦一下

    posted @ 2016-05-17 15:22 小馬歌 閱讀(897) | 評(píng)論 (0)編輯 收藏
     
    from:http://www.infoq.com/cn/articles/Kubernetes-system-architecture-introduction

    1. 前言

    Together we will ensure that Kubernetes is a strong and open container management framework for any application and in any environment, whether in a private, public or hybrid cloud.

    Urs Hölzle, Google

    Kubernetes作為Docker生態(tài)圈中重要一員,是Google多年大規(guī)模容器管理技術(shù)的開(kāi)源版本,是產(chǎn)線實(shí)踐經(jīng)驗(yàn)的最佳表現(xiàn)[G1] 。如Urs Hölzle所說(shuō),無(wú)論是公有云還是私有云甚至混合云,Kubernetes將作為一個(gè)為任何應(yīng)用,任何環(huán)境的容器管理框架無(wú)處不在。正因?yàn)槿绱耍?目前受到各大巨頭及初創(chuàng)公司的青睞,如Microsoft、VMWare、Red Hat、CoreOS、Mesos等,紛紛加入給Kubernetes貢獻(xiàn)代碼。隨著Kubernetes社區(qū)及各大廠商的不斷改進(jìn)、發(fā)展,Kuberentes將成為容器管理領(lǐng)域的領(lǐng)導(dǎo)者。

    接下來(lái)我們會(huì)用一系列文章逐一探索Kubernetes是什么、能做什么以及怎么做。

    2. 什么是Kubernetes

    Kubernetes是Google開(kāi)源的容器集群管理系統(tǒng),其提供應(yīng)用部署、維護(hù)、 擴(kuò)展機(jī)制等功能,利用Kubernetes能方便地管理跨機(jī)器運(yùn)行容器化的應(yīng)用,其主要功能如下:

    1) 使用Docker對(duì)應(yīng)用程序包裝(package)、實(shí)例化(instantiate)、運(yùn)行(run)。

    2) 以集群的方式運(yùn)行、管理跨機(jī)器的容器。

    3) 解決Docker跨機(jī)器容器之間的通訊問(wèn)題。

    4) Kubernetes的自我修復(fù)機(jī)制使得容器集群總是運(yùn)行在用戶(hù)期望的狀態(tài)。

    當(dāng)前Kubernetes支持GCE、vShpere、CoreOS、OpenShift、Azure等平臺(tái),除此之外,也可以直接運(yùn)行在物理機(jī)上。

    接下來(lái)本文主要從以下幾方面闡述Kubernetes:

    1) Kubernetes的主要概念。

    2) Kubernetes的構(gòu)件,包括Master組件、Kubelet、Proxy的詳細(xì)介紹。

    3. Kubernetes主要概念

    3.1. Pods

    Pod是Kubernetes的基本操作單元,把相關(guān)的一個(gè)或多個(gè)容器構(gòu)成一個(gè)Pod,通常Pod里的容器運(yùn)行相同的應(yīng)用。Pod包含的容器運(yùn)行在同一個(gè)Minion(Host)上,看作一個(gè)統(tǒng)一管理單元,共享相同的volumes和network namespace/IP和Port空間。

    3.2. Services

    Services也是Kubernetes的基本操作單元,是真實(shí)應(yīng)用服務(wù)的抽象,每一個(gè)服務(wù)后面都有很多對(duì)應(yīng)的容器來(lái)支持,通過(guò)Proxy的port和服務(wù)selector決定服務(wù)請(qǐng)求傳遞給后端提供服務(wù)的容器,對(duì)外表現(xiàn)為一個(gè)單一訪問(wèn)接口,外部不需要了解后端如何運(yùn)行,這給擴(kuò)展或維護(hù)后端帶來(lái)很大的好處。

    3.3. Replication Controllers

    Replication Controller確保任何時(shí)候Kubernetes集群中有指定數(shù)量的pod副本(replicas)在運(yùn)行, 如果少于指定數(shù)量的pod副本(replicas),Replication Controller會(huì)啟動(dòng)新的Container,反之會(huì)殺死多余的以保證數(shù)量不變。Replication Controller使用預(yù)先定義的pod模板創(chuàng)建pods,一旦創(chuàng)建成功,pod 模板和創(chuàng)建的pods沒(méi)有任何關(guān)聯(lián),可以修改pod 模板而不會(huì)對(duì)已創(chuàng)建pods有任何影響,也可以直接更新通過(guò)Replication Controller創(chuàng)建的pods。對(duì)于利用pod 模板創(chuàng)建的pods,Replication Controller根據(jù)label selector來(lái)關(guān)聯(lián),通過(guò)修改pods的label可以刪除對(duì)應(yīng)的pods。Replication Controller主要有如下用法:

    1) Rescheduling

    如上所述,Replication Controller會(huì)確保Kubernetes集群中指定的pod副本(replicas)在運(yùn)行, 即使在節(jié)點(diǎn)出錯(cuò)時(shí)。

    2) Scaling

    通過(guò)修改Replication Controller的副本(replicas)數(shù)量來(lái)水平擴(kuò)展或者縮小運(yùn)行的pods。

    3) Rolling updates

    Replication Controller的設(shè)計(jì)原則使得可以一個(gè)一個(gè)地替換pods來(lái)rolling updates服務(wù)。

    4) Multiple release tracks

    如果需要在系統(tǒng)中運(yùn)行multiple release的服務(wù),Replication Controller使用labels來(lái)區(qū)分multiple release tracks。

    3.4. Labels

    Labels是用于區(qū)分Pod、Service、Replication Controller的key/value鍵值對(duì),Pod、Service、 Replication Controller可以有多個(gè)label,但是每個(gè)label的key只能對(duì)應(yīng)一個(gè)value。Labels是Service和Replication Controller運(yùn)行的基礎(chǔ),為了將訪問(wèn)Service的請(qǐng)求轉(zhuǎn)發(fā)給后端提供服務(wù)的多個(gè)容器,正是通過(guò)標(biāo)識(shí)容器的labels來(lái)選擇正確的容器。同樣,Replication Controller也使用labels來(lái)管理通過(guò)pod 模板創(chuàng)建的一組容器,這樣Replication Controller可以更加容易,方便地管理多個(gè)容器,無(wú)論有多少容器。

    4. Kubernetes構(gòu)件

    Kubenetes整體框架如下圖3-1,主要包括kubecfg、Master API Server、Kubelet、Minion(Host)以及Proxy。

    圖3-1 Kubernetes High Level構(gòu)件

    4.1. Master

    Master定義了Kubernetes 集群Master/API Server的主要聲明,包括Pod Registry、Controller Registry、Service Registry、Endpoint Registry、Minion Registry、Binding Registry、RESTStorage以及Client, 是client(Kubecfg)調(diào)用Kubernetes API,管理Kubernetes主要構(gòu)件Pods、Services、Minions、容器的入口。Master由API Server、Scheduler以及Registry等組成。從下圖3-2可知Master的工作流主要分以下步驟:

    1) Kubecfg將特定的請(qǐng)求,比如創(chuàng)建Pod,發(fā)送給Kubernetes Client。

    2) Kubernetes Client將請(qǐng)求發(fā)送給API server。

    3) API Server根據(jù)請(qǐng)求的類(lèi)型,比如創(chuàng)建Pod時(shí)storage類(lèi)型是pods,然后依此選擇何種REST Storage API對(duì)請(qǐng)求作出處理。

    4) REST Storage API對(duì)的請(qǐng)求作相應(yīng)的處理。

    5) 將處理的結(jié)果存入高可用鍵值存儲(chǔ)系統(tǒng)Etcd中。

    6) 在API Server響應(yīng)Kubecfg的請(qǐng)求后,Scheduler會(huì)根據(jù)Kubernetes Client獲取集群中運(yùn)行Pod及Minion信息。

    7) 依據(jù)從Kubernetes Client獲取的信息,Scheduler將未分發(fā)的Pod分發(fā)到可用的Minion節(jié)點(diǎn)上。

    下面是Master的主要構(gòu)件的詳細(xì)介紹:

    圖3-2 Master主要構(gòu)件及工作流

    3.1.1. Minion Registry

    Minion Registry負(fù)責(zé)跟蹤Kubernetes 集群中有多少M(fèi)inion(Host)。Kubernetes封裝Minion Registry成實(shí)現(xiàn)Kubernetes API Server的RESTful API接口REST,通過(guò)這些API,我們可以對(duì)Minion Registry做Create、Get、List、Delete操作,由于Minon只能被創(chuàng)建或刪除,所以不支持Update操作,并把Minion的相關(guān)配置信息存儲(chǔ)到etcd。除此之外,Scheduler算法根據(jù)Minion的資源容量來(lái)確定是否將新建Pod分發(fā)到該Minion節(jié)點(diǎn)。

    3.1.2. Pod Registry

    Pod Registry負(fù)責(zé)跟蹤Kubernetes集群中有多少Pod在運(yùn)行,以及這些Pod跟Minion是如何的映射關(guān)系。將Pod Registry和Cloud Provider信息及其他相關(guān)信息封裝成實(shí)現(xiàn)Kubernetes API Server的RESTful API接口REST。通過(guò)這些API,我們可以對(duì)Pod進(jìn)行Create、Get、List、Update、Delete操作,并將Pod的信息存儲(chǔ)到etcd中,而且可以通過(guò)Watch接口監(jiān)視Pod的變化情況,比如一個(gè)Pod被新建、刪除或者更新。

    3.1.3. Service Registry

    Service Registry負(fù)責(zé)跟蹤Kubernetes集群中運(yùn)行的所有服務(wù)。根據(jù)提供的Cloud Provider及Minion Registry信息把Service Registry封裝成實(shí)現(xiàn)Kubernetes API Server需要的RESTful API接口REST。利用這些接口,我們可以對(duì)Service進(jìn)行Create、Get、List、Update、Delete操作,以及監(jiān)視Service變化情況的watch操作,并把Service信息存儲(chǔ)到etcd。

    3.1.4. Controller Registry

    Controller Registry負(fù)責(zé)跟蹤Kubernetes集群中所有的Replication Controller,Replication Controller維護(hù)著指定數(shù)量的pod 副本(replicas)拷貝,如果其中的一個(gè)容器死掉,Replication Controller會(huì)自動(dòng)啟動(dòng)一個(gè)新的容器,如果死掉的容器恢復(fù),其會(huì)殺死多出的容器以保證指定的拷貝不變。通過(guò)封裝Controller Registry為實(shí)現(xiàn)Kubernetes API Server的RESTful API接口REST, 利用這些接口,我們可以對(duì)Replication Controller進(jìn)行Create、Get、List、Update、Delete操作,以及監(jiān)視Replication Controller變化情況的watch操作,并把Replication Controller信息存儲(chǔ)到etcd。

    3.1.5. Endpoints Registry

    Endpoints Registry負(fù)責(zé)收集Service的endpoint,比如Name:"mysql",Endpoints: ["10.10.1.1:1909","10.10.2.2:8834"],同Pod Registry,Controller Registry也實(shí)現(xiàn)了Kubernetes API Server的RESTful API接口,可以做Create、Get、List、Update、Delete以及watch操作。

    3.1.6. Binding Registry

    Binding包括一個(gè)需要綁定Pod的ID和Pod被綁定的Host,Scheduler寫(xiě)B(tài)inding Registry后,需綁定的Pod被綁定到一個(gè)host。Binding Registry也實(shí)現(xiàn)了Kubernetes API Server的RESTful API接口,但Binding Registry是一個(gè)write-only對(duì)象,所有只有Create操作可以使用, 否則會(huì)引起錯(cuò)誤。

    3.1.7. Scheduler

    Scheduler收集和分析當(dāng)前Kubernetes集群中所有Minion節(jié)點(diǎn)的資源(內(nèi)存、CPU)負(fù)載情況,然后依此分發(fā)新建的Pod到Kubernetes集群中可用的節(jié)點(diǎn)。由于一旦Minion節(jié)點(diǎn)的資源被分配給Pod,那這些資源就不能再分配給其他Pod, 除非這些Pod被刪除或者退出, 因此,Kubernetes需要分析集群中所有Minion的資源使用情況,保證分發(fā)的工作負(fù)載不會(huì)超出當(dāng)前該Minion節(jié)點(diǎn)的可用資源范圍。具體來(lái)說(shuō),Scheduler做以下工作:

    1) 實(shí)時(shí)監(jiān)測(cè)Kubernetes集群中未分發(fā)的Pod。

    2) 實(shí)時(shí)監(jiān)測(cè)Kubernetes集群中所有運(yùn)行的Pod,Scheduler需要根據(jù)這些Pod的資源狀況安全地將未分發(fā)的Pod分發(fā)到指定的Minion節(jié)點(diǎn)上。

    3) Scheduler也監(jiān)測(cè)Minion節(jié)點(diǎn)信息,由于會(huì)頻繁查找Minion節(jié)點(diǎn),Scheduler會(huì)緩存一份最新的信息在本地。

    4) 最后,Scheduler在分發(fā)Pod到指定的Minion節(jié)點(diǎn)后,會(huì)把Pod相關(guān)的信息Binding寫(xiě)回API Server。

    4.2. Kubelet

    圖3-3 Kubernetes詳細(xì)構(gòu)件

    根據(jù)上圖3-3可知Kubelet是Kubernetes集群中每個(gè)Minion和Master API Server的連接點(diǎn),Kubelet運(yùn)行在每個(gè)Minion上,是Master API Server和Minion之間的橋梁,接收Master API Server分配給它的commands和work,與持久性鍵值存儲(chǔ)etcd、file、server和http進(jìn)行交互,讀取配置信息。Kubelet的主要工作是管理Pod和容器的生命周期,其包括Docker Client、Root Directory、Pod Workers、Etcd Client、Cadvisor Client以及Health Checker組件,具體工作如下:

    1) 通過(guò)Worker給Pod異步運(yùn)行特定的Action。

    2) 設(shè)置容器的環(huán)境變量。

    3) 給容器綁定Volume。

    4) 給容器綁定Port。

    5) 根據(jù)指定的Pod運(yùn)行一個(gè)單一容器。

    6) 殺死容器。

    7) 給指定的Pod創(chuàng)建network 容器。

    8) 刪除Pod的所有容器。

    9) 同步Pod的狀態(tài)。

    10) 從Cadvisor獲取container info、 pod info、root info、machine info。

    11) 檢測(cè)Pod的容器健康狀態(tài)信息。

    12) 在容器中運(yùn)行命令。

    4.3. Proxy

    Proxy是為了解決外部網(wǎng)絡(luò)能夠訪問(wèn)跨機(jī)器集群中容器提供的應(yīng)用服務(wù)而設(shè)計(jì)的,從上圖3-3可知Proxy服務(wù)也運(yùn)行在每個(gè)Minion上。Proxy提供TCP/UDP sockets的proxy,每創(chuàng)建一種Service,Proxy主要從etcd獲取Services和Endpoints的配置信息,或者也可以從file獲取,然后根據(jù)配置信息在Minion上啟動(dòng)一個(gè)Proxy的進(jìn)程并監(jiān)聽(tīng)相應(yīng)的服務(wù)端口,當(dāng)外部請(qǐng)求發(fā)生時(shí),Proxy會(huì)根據(jù)Load Balancer將請(qǐng)求分發(fā)到后端正確的容器處理。

    5. 下篇主題

    下篇講述在CentOS7上用Kubernetes來(lái)管理容器。

    6. 個(gè)人簡(jiǎn)介

    楊章顯,現(xiàn)就職于Cisco,主要從事WebEx SaaS服務(wù)運(yùn)維,系統(tǒng)性能分析等工作。特別關(guān)注云計(jì)算,自動(dòng)化運(yùn)維,部署等技術(shù),尤其是Go、OpenvSwitch、Docker及其生態(tài)圈技術(shù),如Kubernetes、Flocker等Docker相關(guān)開(kāi)源項(xiàng)目。Email: yangzhangxian@gmail.com

    7. 參考資料

    1. https://github.com/GoogleCloudPlatform/kubernetes/tree/master/docs
    2. http://www.slideshare.net/rajdeep
    3. http://www.docker.com

    感謝郭蕾對(duì)本文的策劃和審校。

    給InfoQ中文站投稿或者參與內(nèi)容翻譯工作,請(qǐng)郵件至editors@cn.infoq.com。也歡迎大家通過(guò)新浪微博(@InfoQ)或者騰訊微博(@InfoQ)關(guān)注我們,并與我們的編輯和其他讀者朋友交流。

    QCon北京2016】近日,極客邦科技經(jīng)常被熱門(mén)大片兒《QCon的后裔》攻陷,不少Q(mào)迷們都直呼膜拜美國(guó)硅谷 Airbnb 美女工程師朱赟的“撩漢”技能,女神已放話要美美的來(lái)QCon啦!男神阿里巴巴資深總監(jiān)莊卓然也將在此邂逅。4月21~23日,3天的約會(huì),等你來(lái)見(jiàn)證。再不行動(dòng),神都幫不了你了。約約約
    posted @ 2016-04-23 11:14 小馬歌 閱讀(346) | 評(píng)論 (0)編輯 收藏
     

    Java Collections Cheat Sheet

    Java Collections cheat sheet

    Every Java program tends to have one thing in common. They’ll all use Java collections! They’re so fundamental, we could not even avoid thinking to omit them from out RebelLabs cheat sheet collection. This is a tough challenge, since there’s so much you need to know about the collections framework, the implementation details, correct use cases, how to choose the right collection type, what can they do and when to turn to the third party libraries as opposed to using the built in collections in the JDK.

    However, we never take the easy path, remember, we’ve tackled:

    Anyway, no topic as broad as Java collections framework can be fully explained in a single A4 page, but we’ve tried to incorporate the most essential information you will need to reference again and again. The corresponding explanations and details behind the decisions are right here, in this blogpost.

    GET ME A JAVA COLLECTIONS CHEAT SHEET!

    We’ve also created an interactive flow of questions that you can go through to determine which Java collection you should consider using for your data. Hit Enter to get going or Esc to go back a step:

    Continue reading to get a better understanding of the collection classes available in Java and when to use them.

    Collections interfaces and implementations in JDK

    You might think that the collections framework was always a part of the JDK. Surprise, surprise, that’s not true. Collections are one of the most fundamentals parts of the JDK, or any programming language for that matter, but it’s also one of the hardest aspects of programming to get right. On top of that it’s really challenging to make the collections library simple, coherent, and easy to use (I’m looking at you, Scala). So the Java collections library was introduced to Java in the Java 1.2 in 1998 and has stuck with us since then. Since backwards compatibility is one of the core values of the Java platform, collections haven’t changed a lot since then. However, recently in the Java 8 release, the language evolved enough to provide tools to enhance the collection interfaces without breaking backwards compatibility.

    All in all, Java collections is a comprehensive library that contains the utilities to tackle almost any container-like data structure you might need. Let’s dive into which types of collections are available and outline what makes them special and why you might want to use them.

    Short description of the most used collections

    There are two main approaches to choosing the collection type you need in your code. The first is simple enough, but not always the most effective:

    if it fits into an ArrayList, use ArrayList, otherwise you probably need a HashMap.

    The other includes having an understanding of the operations you will be needing from the collection, the performance you would expect from them and what kind of data you intend to put into the collection. While the advice above can take you quite far, it’s much better to understand the options available to help you make an informed decision.

    Collection interfaces you need to master

    There are 4 main types of Java collections available. Naturally, the implementations for each type can be found under the corresponding interfaces. Before we start digging into these, there are 2 main umbrella interfaces you want to know about: Iterable and Collection. Typically you rarely need to think about these, but it’s important to at least understand them. The iterable interface allows one to obtain an iterator and traverse the sequence of elements by calling the next() method. It also makes it possible to iterate through elements with the syntactic sugar of the “for-each” loop:

    for(E e: iterable)  

    The collection interface extends the Iterable and represents an iterable group of elements which can be added, removed, or checked for presence in the collection. However, the implementations usually implement the more specific interfaces that are designed with the collection implementation in mind. Here are the most used collection interfaces.

    List

    List is an ordered collection of elements. Some languages call that a sequence, or you can think of if it as an array of varying length. You can add elements to it, in the middle of it, access and replace the elements using an index.

    Not surprisingly, the list is one of the most common collections used. However, there’s almost no variety in the used implementations required. Yeah, in school you learned about the dreaded linked lists and how to implement them and how awesome it is to have access to the head and the tail of it. In practice, you are better using the ArrayList 99% of the time.

    The reason for that is quite simple, it consumes less memory, it is friendlier to the caches, and it is in general faster than the LinkedList from the JDK. In fact, you should try not to use a LinkedList.

    If you foresee that you’ll need to establish concurrent access to a list, you’ll need to worry about the synchronization yourself. Typically, that means that either you will use a CopyOnWriteArrayListwhich is thread-safe and immutable. If your data can be mutated by another thread, but the performance of the reading accesses is much more important. CopyOnWriteArrayList is your list of choice.

    Set

    A collection that contains no duplicate elements is a set. Just like the mathematical set, the interface denotes a collection that holds elements and basically answers a single question: is a given element contained in the set.

    Set doesn’t necessarily specify the order of elements when you iterate them, but all set implementations are created with the performance of the contains method in mind.

    There are two main Set implementations that you should know about: HashSet and TreeSet. HashSet hashes the elements and distributes them into buckets by the hash value. Tree set is backed by a balanced tree, which makes it ordered and navigable (so you can ask what for the previous and next elements by value). TreeSet operations will have worse complexity compared to the HashSet as a result, but bear in mind that the operations still take sublinear time of the set size, which means that for realistic values of the set sizes, it’s still quite a short time.

    However, typically use-cases for sets do not require navigating from element to element, so the HashSet is the goto implementation you’ll tend to use. And it’s a good one. Just remember to correctly implement the hashCode() and equals() methods for your elements.

    Map

    Perhaps the most used collection type of all time — an object that maps keys to values, an associative array, a table: the map. Maps are the most versatile collection type because the association of keys to values is really what computers are all about: a dictionary, mapping object properties to their values just like javascript does, mapping filenames to their contents and metadata, mapping usernames to password hashes, session attributes, product cart items, game high scores. Wherever you look you’ll find a use case for a map.

    Naturally there are different implementations that are tweaked to provide different performance trade-offs. But the default goto implementation of the map interface is undoubtedly the infamousHashMap.

    The Hashmap implementation depends on the Key objects implementing the hashCode() and equals() methods correctly, so always take care of these. In return the HashMap promises you the almost constant time performance which is as amazing as you can get out of any data structure and scales amazingly well.

    If you need the navigation between elements or you’d need to deal with unhashable element types, you can use a TreeMap. Using a balanced tree, just like the sets above, treemap scales appropriately and gives you a chance to iterate the contents in a sorted order.

    Now we don’t want to dive into all the API methods for maps. You can put and get values, as you’d expect. But we would love to point out one super amazing method that has been available to us since Java 8: computeIfAbsent.

    default V computeIfAbsent(K key,                           Function<? super K,? extends V> mappingFunction) 

    What computeIfAbsent does is give you an opportunity to specify how to obtain the value for the key, if it’s not in the collection yet. Essentially it checks if the given key is contained in the map. If it happens to be so, you get the corresponding value back. If not, the mapping function is executed and the resulting values is put into the map and returned to you.

    That’s more or less how caches work. Amazing, right?

    Queue

    queue is a collection designed to hold elements prior to processing, the elements wait in line to be consumed and can be added to the tail of the queue. Queues are those fundamental pieces of software that tie components together. When components are communicating in a system, there’s typically a queue of messages sitting between them.

    Queues are a special sort of collection, because are mostly operated using the Queue interface, rather than methods inherited from the Collection interface. To add an element to a queue, use:

    E e;  Queue<E> q = …  q.offer(e); // try to put the element into the queue, if the queue is full, do nothing q.poll(); // remove the head of the queue  q.peek(); // return the head of the queue without modifying it. 

    There are also corresponding methods that throw exceptions if operations don’t succeed. But all in all, there are three main operations one performs on a queue: enqueue, deque and peek.
    Note that you can get an iterator from the queue and process the elements similar to any other collection. It would even work for some implementations of the queue interface, but not for all.

    These are the typical classes that you might use that implement the queue interface:

    • ArrayDeque – a good general purpose battle-tested implementation of a queue. ArrayDeque is backed by a resizeable array that can grow to accommodate additional elements. Most ArrayDeque operations run in amortized constant time.
    • PriorityQueue – a queue of sorted elements. The sorting is achieved either naturally or determined by a provided comparator, but the head of the queue is always the minimal element. Note that a priority queue won’t change its order if you mutate the elements inside it. If you’re curious about the performance, the mutating operations run in logarithmic time of the queue size, peeking and such — in constant, and the search is linear.

    Collection utilities

    Now we’ve talked about all the most common collection types you’ll encounter, it’s time to dig into the other parts of the Java collections framework. There’s an amazing class, calledCollections, that provides handy utility methods that you should be aware of. First of all let’s talk about the collection wrappers. Any collection type can have additional requirements to its functionality, these orthogonal qualities are implemented with the wrappers. The original collection is wrapped into another one that delegates the element handling logic to it.

    The prime example of some functionality that is suitable for a wrapper is making a collection immutable. Indeed, you just need to ignore all the mutating operations like: put, insert, and so on. And all the non-mutating operations can be easily delegated to the actual collection implementations.

    The Collections class provides a set of methods that give you just that: unmodifiable collections:

    Java-collections-wrappers

    Another useful type of wrapper available is the synchronized collections wrapper. Indeed, most of the collection implementations do not implement the synchronization themselves. What would you do if you need to use HashSet concurrently from multiple threads? That’s right, you call the corresponding wrapper method and use the returned set.

    Collections.synchronizedSet(new HashSet<E>()); 

    It’s important to remember that the wrapper does not change the implementation of the underlying collection. That is if you obtain a reference to the wrapped hash set and use that concurrently there are no guarantees.

    Here’s a short list of the useful methods in the Collections class that typically are used more than others:

    • binarySearch – a fast way to find an element in a sorted List.
    • frequency – tells you how many times an element is encountered in a collection
    • min / max – returns the smallest / largest element of the collection
    • reverseOrder – provides you with a Comparator to sort elements in the descending order
    • singleton – wraps an object into the Set containing that object, the similar methods are available for the List and Map classes.

    Other libraries

    There are more questions that you can ask of your collection implementation and obviously there are more answers than just the stock implementations you can find in the JDK. Here are some libraries that we think are amazing and give you more functionality for the collection creation and manipulation and more collection types.

    • Guava – Google Core Libraries for Java 6+. Perhaps the default third party collection library for Java projects. Contains a magnitude of convenient methods for creating collection, like fluent builders, as well as advanced collection types.
    • Eclipse Collections – Features you want with the collections you need. Previously known as gs-collections, this library includes almost any collection you might need: primitive type collections, multimaps, bidirectional maps and so on.
    • Fastutil – Fast & compact type-specific collections for Java. Great default choice for collections of primitive types, like int or long. Also handles big collections with more than 2^31 elements.
    • JCTools – Java Concurrency Tools for the JVM. If you work on high throughput concurrent applications and need a way to increase your performance, check out JCTools. It contains lots of queues for all kinds of single / multiple producer / consumer environments and other useful classes.

    Sure, this is not a definitive list, but these are the essential libraries that you should know about. You’re might be using Guava in your project already; at least once you have thought about how awesome would it be to have the collections for primitives, and so on. So check these out!

    Improving JDK collections

    It’s not easy to change something as fundamental as the collections library in the JDK. However, the Java 8 release had implemented a major enhancements to the collections library and more than that, with the inclusion of the default methods it made it possible to evolve the collection interfaces without breaking all of the implementations.

    The progress doesn’t stop there. The upcoming Java 9 release contains the JEP 269: Convenience Factory Methods for Collections, the proposal to include the methods for easier collection creation. Finally, we would be able to specify the collections with a simpler syntax like:

      List.of(a, b, c); Set.of(d, e, f, g); Map.ofEntries(     entry(k1, v1),     entry(k2, v2),     entry(k3, v3)); 

    That would be really sweet.

    Conclusion

    We’ve talked a lot about Java collections and different implementations of the collection interfaces you can find in the java.util package. Sets, Lists and Maps are your friends and you’ll need to apply quite a bit of imagination to come up with a program that doesn’t need the collections API. We touched on the subject of collection wrappers and how to make collections immutable and synchronize them for concurrent access.

    Be sure to check out those libraries above with the alternative collections implementations. Hopefully you liked this post and the cheat sheet that tries to give you the information about Java collections on a single printable A4 sized piece of paper.

    SHOW ME JAVA COLLECTIONS CHEAT SHEET!

      posted @ 2016-04-22 14:46 小馬歌 閱讀(494) | 評(píng)論 (0)編輯 收藏
       
           摘要: 時(shí)間 2013-04-16 09:08:44  Script Ahead, Code Behind原文  http://rednaxelafx.iteye.com/blog/1847971主題 JVM(未經(jīng)許可請(qǐng)勿轉(zhuǎn)載。希望轉(zhuǎn)載請(qǐng)與我聯(lián)系。) (如果打開(kāi)此頁(yè)面時(shí)瀏覽器有點(diǎn)卡住的話請(qǐng)耐心等待片刻。大概是ItEye的代碼高亮太耗時(shí)了...  閱讀全文
      posted @ 2016-04-22 09:59 小馬歌 閱讀(590) | 評(píng)論 (0)編輯 收藏
       
      作者:曹旭東
      鏈接:https://www.zhihu.com/question/24401191/answer/37601385
      來(lái)源:知乎
      著作權(quán)歸作者所有。商業(yè)轉(zhuǎn)載請(qǐng)聯(lián)系作者獲得授權(quán),非商業(yè)轉(zhuǎn)載請(qǐng)注明出處。

      下文所使用的java版本信息

      $ java -version java version "1.8.0_20" Java(TM) SE Runtime Environment (build 1.8.0_20-b26) Java HotSpot(TM) 64-Bit Server VM (build 25.20-b23, mixed mode) 


      java中的注解是一種繼承自接口`java.lang.annotation.Annotation`的特殊接口。參見(jiàn)下面的[JLS][2]和JDK文檔。

      An annotation type declaration specifies a new annotation type,  a special kind of interface type. To distinguish an annotation  type declaration from a normal interface declaration, the keyword  interface is preceded by an at-sign (@).  ...  The direct superinterface of every annotation type is java.lang.annotation.Annotation. 

      /**  * The common interface extended by all annotation types.  Note that an  * interface that manually extends this one does <i>not</i> define  * an annotation type.  Also note that this interface does not itself  * define an annotation type.  *  * More information about annotation types can be found in section 9.6 of  * <cite>The Java&trade; Language Specification</cite>.  *  * The {@link java.lang.reflect.AnnotatedElement} interface discusses  * compatibility concerns when evolving an annotation type from being  * non-repeatable to being repeatable.  *  * @author  Josh Bloch  * @since   1.5  */ public interface Annotation {     ... } 

      下面看一下具體示例。

      定義一個(gè)注解:
      import java.lang.annotation.ElementType; import java.lang.annotation.Retention; import java.lang.annotation.RetentionPolicy; import java.lang.annotation.Target;  /**  * Created by Administrator on 2015/1/18.  */ @Target(ElementType.TYPE) @Retention(RetentionPolicy.RUNTIME) public @interface TestAnnotation {      int count() default 1;  } 

      經(jīng)過(guò)編譯之后,注解`TestAnnotation`的字節(jié)碼是這樣的:
      Classfile /e:/workspace/intellij/SpringTest/target/classes/TestAnnotation.class   Last modified 2015-1-18; size 379 bytes   MD5 checksum 200dc3a75216b7a88ae17873d5dffd4f   Compiled from "TestAnnotation.java" public interface TestAnnotation extends java.lang.annotation.Annotation   minor version: 0   major version: 52   flags: ACC_PUBLIC, ACC_INTERFACE, ACC_ABSTRACT, ACC_ANNOTATION Constant pool:    #1 = Class              #14            // TestAnnotation    #2 = Class              #15            // java/lang/Object    #3 = Class              #16            // java/lang/annotation/Annotation    #4 = Utf8               SourceFile    #5 = Utf8               TestAnnotation.java    #6 = Utf8               RuntimeVisibleAnnotations    #7 = Utf8               Ljava/lang/annotation/Target;    #8 = Utf8               value    #9 = Utf8               Ljava/lang/annotation/ElementType;   #10 = Utf8               TYPE   #11 = Utf8               Ljava/lang/annotation/Retention;   #12 = Utf8               Ljava/lang/annotation/RetentionPolicy;   #13 = Utf8               RUNTIME   #14 = Utf8               TestAnnotation   #15 = Utf8               java/lang/Object   #16 = Utf8               java/lang/annotation/Annotation { } SourceFile: "TestAnnotation.java" RuntimeVisibleAnnotations:   0: #7(#8=[e#9.#10])   1: #11(#8=e#12.#13) 

      從反編譯后的信息中可以看出,注解就是一個(gè)繼承自`java.lang.annotation.Annotation`的接口。

      那么接口怎么能夠設(shè)置屬性呢?簡(jiǎn)單來(lái)說(shuō)就是java通過(guò)動(dòng)態(tài)代理的方式為你生成了一個(gè)實(shí)現(xiàn)了"接口"`TestAnnotation`的實(shí)例(對(duì)于當(dāng)前的實(shí)體來(lái)說(shuō),例如類(lèi)、方法、屬性域等,這個(gè)代理對(duì)象是單例的),然后對(duì)該代理實(shí)例的屬性賦值,這樣就可以在程序運(yùn)行時(shí)(如果將注解設(shè)置為運(yùn)行時(shí)可見(jiàn)的話)通過(guò)反射獲取到注解的配置信息。

      具體來(lái)說(shuō)是怎么實(shí)現(xiàn)的呢?

      寫(xiě)一個(gè)使用該注解的類(lèi):
      import java.io.IOException;  /**  * Created by Administrator on 2015/1/18.  */ @TestAnnotation(count = 0x7fffffff) public class TestMain {      public static void main(String[] args) throws InterruptedException, NoSuchFieldException, IllegalAccessException, IOException {         TestAnnotation annotation = TestMain.class.getAnnotation(TestAnnotation.class);         System.out.println(annotation.count());         System.in.read();     }  } 

      反編譯一下這段代碼:
      Classfile /e:/workspace/intellij/SpringTest/target/classes/TestMain.class   Last modified 2015-1-20; size 1006 bytes   MD5 checksum a2d5367ea568240f078d5fb1de917550   Compiled from "TestMain.java" public class TestMain   minor version: 0   major version: 52   flags: ACC_PUBLIC, ACC_SUPER Constant pool:    #1 = Methodref          #10.#34        // java/lang/Object."<init>":()V    #2 = Class              #35            // TestMain    #3 = Class              #36            // TestAnnotation    #4 = Methodref          #37.#38        // java/lang/Class.getAnnotation:(Ljava/lang/Class;)Ljava/lang/annotation/Annotation;    #5 = Fieldref           #39.#40        // java/lang/System.out:Ljava/io/PrintStream;    #6 = InterfaceMethodref #3.#41         // TestAnnotation.count:()I    #7 = Methodref          #42.#43        // java/io/PrintStream.println:(I)V    #8 = Fieldref           #39.#44        // java/lang/System.in:Ljava/io/InputStream;    #9 = Methodref          #45.#46        // java/io/InputStream.read:()I   #10 = Class              #47            // java/lang/Object   #11 = Utf8               <init>   #12 = Utf8               ()V   #13 = Utf8               Code   #14 = Utf8               LineNumberTable   #15 = Utf8               LocalVariableTable   #16 = Utf8               this   #17 = Utf8               LTestMain;   #18 = Utf8               main   #19 = Utf8               ([Ljava/lang/String;)V   #20 = Utf8               args   #21 = Utf8               [Ljava/lang/String;   #22 = Utf8               annotation   #23 = Utf8               LTestAnnotation;   #24 = Utf8               Exceptions   #25 = Class              #48            // java/lang/InterruptedException   #26 = Class              #49            // java/lang/NoSuchFieldException   #27 = Class              #50            // java/lang/IllegalAccessException   #28 = Class              #51            // java/io/IOException   #29 = Utf8               SourceFile   #30 = Utf8               TestMain.java   #31 = Utf8               RuntimeVisibleAnnotations   #32 = Utf8               count   #33 = Integer            2147483647   #34 = NameAndType        #11:#12        // "<init>":()V   #35 = Utf8               TestMain   #36 = Utf8               TestAnnotation   #37 = Class              #52            // java/lang/Class   #38 = NameAndType        #53:#54        // getAnnotation:(Ljava/lang/Class;)Ljava/lang/annotation/Annotation;   #39 = Class              #55            // java/lang/System   #40 = NameAndType        #56:#57        // out:Ljava/io/PrintStream;   #41 = NameAndType        #32:#58        // count:()I   #42 = Class              #59            // java/io/PrintStream   #43 = NameAndType        #60:#61        // println:(I)V   #44 = NameAndType        #62:#63        // in:Ljava/io/InputStream;   #45 = Class              #64            // java/io/InputStream   #46 = NameAndType        #65:#58        // read:()I   #47 = Utf8               java/lang/Object   #48 = Utf8               java/lang/InterruptedException   #49 = Utf8               java/lang/NoSuchFieldException   #50 = Utf8               java/lang/IllegalAccessException   #51 = Utf8               java/io/IOException   #52 = Utf8               java/lang/Class   #53 = Utf8               getAnnotation   #54 = Utf8               (Ljava/lang/Class;)Ljava/lang/annotation/Annotation;   #55 = Utf8               java/lang/System   #56 = Utf8               out   #57 = Utf8               Ljava/io/PrintStream;   #58 = Utf8               ()I   #59 = Utf8               java/io/PrintStream   #60 = Utf8               println   #61 = Utf8               (I)V   #62 = Utf8               in   #63 = Utf8               Ljava/io/InputStream;   #64 = Utf8               java/io/InputStream   #65 = Utf8               read {   public TestMain();     descriptor: ()V     flags: ACC_PUBLIC     Code:       stack=1, locals=1, args_size=1          0: aload_0          1: invokespecial #1                  // Method java/lang/Object."<init>":()V          4: return       LineNumberTable:         line 7: 0       LocalVariableTable:         Start  Length  Slot  Name   Signature             0       5     0  this   LTestMain;    public static void main(java.lang.String[]) throws java.lang.InterruptedException, java.lang.NoSuchFieldException, java.lang.IllegalAccessException, java.io.IOException;     descriptor: ([Ljava/lang/String;)V     flags: ACC_PUBLIC, ACC_STATIC     Code:       stack=2, locals=2, args_size=1          0: ldc           #2                  // class TestMain          2: ldc           #3                  // class TestAnnotation          4: invokevirtual #4                  // Method java/lang/Class.getAnnotation:(Ljava/lang/Class;)Ljava/lang/annotation/Annotation;          7: checkcast     #3                  // class TestAnnotation         10: astore_1         11: getstatic     #5                  // Field java/lang/System.out:Ljava/io/PrintStream;         14: aload_1         15: invokeinterface #6,  1            // InterfaceMethod TestAnnotation.count:()I         20: invokevirtual #7                  // Method java/io/PrintStream.println:(I)V         23: getstatic     #8                  // Field java/lang/System.in:Ljava/io/InputStream;         26: invokevirtual #9                  // Method java/io/InputStream.read:()I         29: pop         30: return       LineNumberTable:         line 10: 0         line 11: 11         line 12: 23         line 13: 30       LocalVariableTable:         Start  Length  Slot  Name   Signature             0      31     0  args   [Ljava/lang/String;            11      20     1 annotation   LTestAnnotation;     Exceptions:       throws java.lang.InterruptedException, java.lang.NoSuchFieldException, java.lang.IllegalAccessException, java.io.IOException } SourceFile: "TestMain.java" RuntimeVisibleAnnotations:   0: #23(#32=I#33) 

      最后一行的代碼說(shuō)明,注解`TestAnnotation`的屬性設(shè)置是在編譯時(shí)就確定了的。(對(duì)屬性的說(shuō)明在[這里][1])。

      然后,運(yùn)行上面的程序,通過(guò)CLHSDB在eden區(qū)找到注解實(shí)例,

      hsdb> scanoops 0x00000000e1b80000 0x00000000e3300000 TestAnnotation 0x00000000e1d6c360 com/sun/proxy/$Proxy1 

      類(lèi)型`com/sun/proxy/$Proxy1`是jdk動(dòng)態(tài)代理生成對(duì)象時(shí)的默認(rèn)類(lèi)型,其中`com.sun.proxy`是默認(rèn)的包名,定義于`ReflectUtil`類(lèi)的`PROXY_PACKAGE`字段中。代理類(lèi)名`$PROXY1`包含兩部分,其中前綴`$PROXY`是jdk種默認(rèn)的代理類(lèi)類(lèi)名前綴(參見(jiàn)`java.lang.reflect.Proxy`類(lèi)的javadoc),后的1是自增的結(jié)果。

      下面看一下這個(gè)代理類(lèi)的內(nèi)容。運(yùn)行java程序時(shí)添加參數(shù)`-Dsun.misc.ProxyGenerator.saveGeneratedFiles=true`可以將轉(zhuǎn)儲(chǔ)出jdk動(dòng)態(tài)代理類(lèi)的class文件。若是項(xiàng)目較大或是使用了各種框架的話,慎用此參數(shù)。

      Classfile /e:/workspace/intellij/SpringTest/target/classes/com/sun/proxy/$Proxy1.class   Last modified 2015-1-19; size 2062 bytes   MD5 checksum 7321e44402258ba9e061275e313c5c9f public final class com.sun.proxy.$Proxy1 extends java.lang.reflect.Proxy implements TestAnnotation   minor version: 0   major version: 49   flags: ACC_PUBLIC, ACC_FINAL ... 

      太長(zhǎng)了,只截取一部分。從中可以看到,這個(gè)代理類(lèi)實(shí)現(xiàn)了繼承自`java.lang.reflect.Proxy`類(lèi),又實(shí)現(xiàn)了“接口”TestAnnotation。

      接下來(lái)查看一下代理對(duì)象的內(nèi)容:

      hsdb> inspect 0x00000000e1d6c360 instance of Oop for com/sun/proxy/$Proxy1 @ 0x00000000e1d6c360 @ 0x00000000e1d6c360 (size = 16) _mark: 1 _metadata._compressed_klass: InstanceKlass for com/sun/proxy/$Proxy1 h: Oop for sun/reflect/annotation/AnnotationInvocationHandler @ 0x00000000e1ce7670 Oop for sun/reflect/annotation/Annota tionInvocationHandler @ 0x00000000e1ce7670 
      其中,0xe1ce74e0是成員變量h的地址(h是定義在`java.lang.reflect.Proxy`類(lèi)中的),通過(guò)查看類(lèi)`AnnotationInvocationHandler`的源碼可以知道注解的代理實(shí)例的值就存儲(chǔ)在它的成員變量`memberValues`中,然后繼續(xù)向下挖就好了:

      hsdb> inspect 0x00000000e1ce7670 instance of Oop for sun/reflect/annotation/AnnotationInvocationHandler @ 0x00000000e1ce7670 @ 0x00000000e1ce7670 (size =  24) _mark: 1 _metadata._compressed_klass: InstanceKlass for sun/reflect/annotation/AnnotationInvocationHandler type: Oop for java/lang/Class @ 0x00000000e1ccc5f8 Oop for java/lang/Class @ 0x00000000e1ccc5f8 memberValues: Oop for java/util/LinkedHashMap @ 0x00000000e1ce7548 Oop for java/util/LinkedHashMap @ 0x00000000e1ce7548 memberMethods: null null hsdb> inspect 0x00000000e1ce7548 instance of Oop for java/util/LinkedHashMap @ 0x00000000e1ce7548 @ 0x00000000e1ce7548 (size = 56) _mark: 1 _metadata._compressed_klass: InstanceKlass for java/util/LinkedHashMap keySet: null null values: null null table: ObjArray @ 0x00000000e1ce75b8 Oop for [Ljava/util/HashMap$Node; @ 0x00000000e1ce75b8 entrySet: null null size: 1 modCount: 1 threshold: 1 loadFactor: 0.75 head: Oop for java/util/LinkedHashMap$Entry @ 0x00000000e1ce75d0 Oop for java/util/LinkedHashMap$Entry @ 0x00000000e1ce7 5d0 tail: Oop for java/util/LinkedHashMap$Entry @ 0x00000000e1ce75d0 Oop for java/util/LinkedHashMap$Entry @ 0x00000000e1ce75d0 accessOrder: false hsdb> inspect 0x00000000e1ce75d0 instance of Oop for java/util/LinkedHashMap$Entry @ 0x00000000e1ce75d0 @ 0x00000000e1ce75d0 (size = 40) _mark: 1 _metadata._compressed_klass: InstanceKlass for java/util/LinkedHashMap$Entry hash: 94852264 key: "count" @ 0x00000000e1bd7c90 Oop for java/lang/String @ 0x00000000e1bd7c90 value: Oop for java/lang/Integer @ 0x00000000e1ce7630 Oop for java/lang/Integer @ 0x00000000e1ce7630 next: null null before: null null after: null null hsdb> inspect 0x00000000e1ce7630 instance of Oop for java/lang/Integer @ 0x00000000e1ce7630 @ 0x00000000e1ce7630 (size = 16) _mark: 1 _metadata._compressed_klass: InstanceKlass for java/lang/Integer value: 2147483647 

      最后可以看到,key=“count”, value=Integer(2147483647 = 0x7fffffff),正是在TestMain中設(shè)置的值.

      嗯,就這樣吧。


      [1]: Chapter 4. The class File Format
      [2]: Chapter 9. Interfaces
      posted @ 2016-04-22 09:58 小馬歌 閱讀(353) | 評(píng)論 (0)編輯 收藏
       
           摘要: from:http://my.oschina.net/pangyangyang/blog/361753ElasticSearch的安裝http://www.elasticsearch.org/下載最新的ElastiSearch版本。解壓下載文件。cd到${esroot}/bin/,執(zhí)行elasticsearch啟動(dòng)。使用curl -XPOST localhost:9200/_shutdown關(guān)閉E...  閱讀全文
      posted @ 2016-04-15 14:03 小馬歌 閱讀(493) | 評(píng)論 (0)編輯 收藏
       

      我安裝的MySQL版本是5.7.10 。 官網(wǎng)最新版本下載地址是:MySQL下載地址

      1、選擇一個(gè)DMG 后  下載->安裝

      安裝完后會(huì)提示一句話 如下

      A temporary password is generated for root@localhost: dwstkti5xJ<5

      If you lose this password, please consult the section How to Reset the Root Password in the MySQL reference manual.

      把root@localhost: dwstkti5xJ<5 復(fù)制到一個(gè)地方 后面要用。

      mysql 默認(rèn)是安裝到了  usr/local/mysql  下面

      2、安裝完后 在偏好設(shè)置里啟動(dòng)MySQL服務(wù),點(diǎn)擊"Start MySQL Server"



      3、添加MySQL的快捷 命令 方式 。 為什么要弄這個(gè)? 為了方便的直接在終端默認(rèn)打開(kāi)的目錄下使用mysql xxx 命令 而不用麻煩的進(jìn)入mysql的安裝目錄下 進(jìn)行操作。 有如下兩個(gè):

      alias mysql=/usr/local/mysql/bin/mysql

      alias mysqladmin=/usr/local/mysql/bin/mysqladmin

      4、重置(修改)MySQL的root密碼。為什么要重置呢?MySQL安裝完后會(huì)給一個(gè)臨時(shí)的密碼 也就是上文中的 dwstkti5xJ<5   ,如果你不修改這個(gè)密碼 使用臨時(shí)密碼登陸mysql 后 各種命令是不能用的,會(huì)一直提示你  需要重置密碼。

      mysqladmin -u root -p password hahaha      #hahaha是我要修改為的密碼  回車(chē)后 輸入臨時(shí)密碼dwstkti5xJ<5   就算是修改成功了

      5、使用 步驟4里設(shè)置的新密碼來(lái)登陸。 為什么要登陸?登陸后才能在mysql里創(chuàng)建數(shù)據(jù)庫(kù)和各種表 等等。

      mysql -u root -p

      輸入新密碼 后 回車(chē)

      6、設(shè)置配置文件。為什么要弄配置文件 ?  其實(shí)到第五步 就算是安裝完成了,但是正常情況下mysql需要一個(gè)配置文件,里面存放了 許多屬性 比如字符編碼啦  連接數(shù)啦 什么的。這個(gè)配置文件默認(rèn)是在 /usr/local/mysql/support-files/  下面  叫做 my-default.cnf 。 mysql啟動(dòng)時(shí)默認(rèn)會(huì)從下面四個(gè)位置尋找my.cnf 然后使用 , 大家都使用第一種情況,所以咱也使用。

       /etc/my.cnf      /etc/mysql/my.cnf      /usr/local/mysql/etc/my.cnf       ~/.my.cnf

      所以現(xiàn)在要做的就是 把my-default.cnf 復(fù)制到 /etc 的下面,并且修改名字為my.cnf 。 命令如下:

      sudo cp -rv /usr/local/mysql/support-files/my-default.cnf  /etc  #復(fù)制 配置文件 到etc下面

      sudo mv my-default.cnf my.cnf    #修改名字為my.cnf

      配置文件就算是弄好了  要想使之生效 必須重啟mysql ,還是到偏好設(shè)置里 先停止 再開(kāi)啟。

      7、修改配置文件my.cnf 。為什么要修改呢?可以不改  本步驟只是根據(jù)一個(gè)例子 說(shuō)明如何修改配置文件。本步驟修改的是字符的編碼。  使用新密碼登陸mysql后 

      mysql> show variables like '%char%';    #回車(chē)后  會(huì) 看到如下文字

      | Variable_name            | Value                                                  |

      | character_set_client    | utf8                                                  |

      | character_set_connection | utf8                                                  |

      | character_set_database  | latin1                                                |

      | character_set_filesystem | binary                                                |

      | character_set_results    | utf8                                                  |

      | character_set_server    | latin1                                                |

      | character_set_system    | utf8                                                  |

      | character_sets_dir      | /usr/local/mysql-5.7.10-osx10.9-x86_64/share/charsets/

      可以看到character_set_server和character_set_database的字符編碼是latin1  我現(xiàn)在就是要把它改為utf8格式的。

      sudo chmod a+w /etc/my.cnf   #修改權(quán)限為可寫(xiě)  因?yàn)閺?fù)制過(guò)來(lái)的這個(gè)my-default.cnf文件(現(xiàn)在改名為my.cnf了) 是只讀權(quán)限的 要想修改里面的內(nèi)容當(dāng)然要改為可寫(xiě)權(quán)限的了。

      vi  /etc/my.cnf  #進(jìn)去后開(kāi)始修改  找到 [mysqld]  這個(gè)標(biāo)示后在它的下面粘貼上需要配置的參數(shù)  最終效果如下


      修改完后 保存退出。

      sudo chmod a-w /etc/my.cnf   #取消my.cnf 的可寫(xiě)權(quán)限  因?yàn)槿绻蝗∠@個(gè)配置文件的可寫(xiě)權(quán)限  mysql啟動(dòng)時(shí)就不理你修改好的這個(gè)配置文件 就是給你忽略掉了,意思就是說(shuō) 這個(gè)配置文件必須是只讀的。

      修改完后再在偏好設(shè)置里重啟一下mysql。

      各種字符編碼的意義如下 (網(wǎng)上抄的)

      character_set_client  為客戶(hù)端使用的字符集;

      character_set_connection 為連接數(shù)據(jù)庫(kù)的字符集設(shè)置類(lèi)型 如果程序沒(méi)有指明連接數(shù)據(jù)庫(kù)使用的字符集類(lèi)型 則按照服務(wù)器端默認(rèn)的字符集設(shè)置。

      character_set_database  為數(shù)據(jù)庫(kù)服務(wù)器中某個(gè)庫(kù)使用的字符集設(shè)定,如果建庫(kù)時(shí)沒(méi)有指明 將使用服務(wù)器安裝時(shí)指定的字符集設(shè)置。

      character_set_results  為數(shù)據(jù)庫(kù) 給客戶(hù)端返回時(shí)使用的字符集設(shè)定 如果沒(méi)指明 使用服務(wù)端默認(rèn)的字符集。

      character_set_server  為服務(wù)器安裝時(shí)指定的默認(rèn)字符集設(shè)定

      character_set_system    為數(shù)據(jù)庫(kù)系統(tǒng)使用的字符集設(shè)定


      結(jié)束。



      文/大象飛(簡(jiǎn)書(shū)作者)
      原文鏈接:http://www.jianshu.com/p/65ee08a4a0d0
      著作權(quán)歸作者所有,轉(zhuǎn)載請(qǐng)聯(lián)系作者獲得授權(quán),并標(biāo)注“簡(jiǎn)書(shū)作者”。
      posted @ 2016-04-15 10:14 小馬歌 閱讀(338) | 評(píng)論 (0)編輯 收藏
       
           摘要: from:http://blog.csdn.net/smartsmile2012/article/details/17316351版權(quán)聲明:本文為博主原創(chuàng)文章,未經(jīng)博主允許不得轉(zhuǎn)載。[html] view plain copy print?<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML&n...  閱讀全文
      posted @ 2016-04-13 13:47 小馬歌 閱讀(343) | 評(píng)論 (0)編輯 收藏
       
           摘要: from:http://blog.csdn.net/isea533/article/details/50412212Spring Boot 靜態(tài)資源處理Spring Boot 默認(rèn)的處理方式就已經(jīng)足夠了,默認(rèn)情況下Spring Boot 使用WebMvcAutoConfiguration中配置的各種屬性。建議使用Spring Boot 默認(rèn)處理方式,需要自己配置的地方可以通過(guò)配置文件修改。但是如果...  閱讀全文
      posted @ 2016-04-12 14:21 小馬歌 閱讀(1036) | 評(píng)論 (0)編輯 收藏
       

      from:http://zeroturnaround.com/rebellabs/5-unexpectedly-useful-command-line-tools-you-might-overlook/

      My previous RebelLabs post showed us that there still is a nice amount of interest among the developers in neat command line tools. This is great news as it means there are still lots of geeks reaching for their maximum l33t potential! This post will, as the title suggests, jump into 5 more tools that will hopefully tickle your fancy.

      Whilst it is hard to find a set of tools relevant for every different command line professional or enthusiast, I can still wholly recommend glancing over these tools briefly. You never know when you’ll get stuck in a terminal only environment or will need to impress a friend with your exceptional command line skills. So without further ado, here are 5 more command line tools you should consider using!

      z

      1. Install it once
      2. Forget it’s even there
      3. Profit from its productivity: z.

      OK, but what’s it all about? Well, z allows you to quickly jump from folder to folder, without having to bother writing the full absolute or relative paths. To achieve this, it simply records all the folders you visit, and then ranks them based on a combination of the frequency and time of last use. Now all you need to do is type z part/of/path and hit enter. With this, z will automagically guide you to the highest ranking match.

      To install z, download z.sh and source it in ~/.zshrc or ~/.bashrc, if you still haven’t made the inevitable jump to ZSH. Now go about your regular flow, cd-ing all over the place, and you are done!

      Alternatively, if you use the oh-my-zsh framework, simply add it in the plugins listing of ~/.zshrc.

       $ vim ~/.zshrc  plugins=(brew git mercurial mvn osx sbt scala vi-mode z) 

      By default, z stores its data in ~/.z, below are my contents after I cleared them and moved around a bit. After printing the data, you can see I moved to 2 different folders by specifying only a small part of the path, irrespective of the current folder. Take note that the directory name was matched mid-word in a case insensitive way — very handy!

      All in all I highly recommend this great tool to anyone that spends any amount of time at the command line, whatever your background, interest or use case may be.

      z

      youtube-dl

      The second tool of this post is the excellent youtube-dl. A cross-platform tool allowing you to, unsuprisingly, download Youtube videos. The easiest way to install is either via Homebrew orpip as shown below. Other download options can be found via the link above.

       $ brew install youtube-dl  $ sudo pip install --upgrade youtube_dl 

      At times, for whatever reason, you may want to download a presentation for archiving’s purposes. Or better yet, imagine you spend the weekend in the Estonian countryside, devoid of internet, yet you really wanted to see the latest Virtual JUG sessions. A simple youtube-dl https://youtu.be/PQPvZkA-6bg suffices at this point.

      youtube-dl

      Other nice things to take note of:

      • You are not limited to downloading from Youtube only, Vimeo for example is also supported.
      • Simply point to a Youtube playlist and youtube-dl will be smart enough to automatically download the full list.
      • As shown in the example image, youtube-dl will happily follow shortened URLs for you.
      • By default the highest quality video will be downloaded, however a simple youtube-dl -f $FORMAT $LINK will override this. You find available formats with the -F flag.
      • A veritable host of additional options are nicely documented and easily accessible via man youtube-dl.

      shellcheck

      I will assume that anyone spending significant some time at the command line will want to automate tasks using shell scripts. Bash itself is renowned for its many pitfalls, and even advanced scripters will from time to time bump into something unexpected. Unfortunately for us, a shell script has no undo button, and “unexpected” results may well be synonymous with “catastrophic” results.

      Long intro short: we have a valuable tool on our side to guard ourselves against exactly this,shellcheck! In essence, it is a static analyser that will tell you where your script goes wrong.

      Install shellcheck via your package manager of choice: Homebrew, Pacman, APT; build it from source, or run it inside your browser.

       $ brew install shellcheck  # pacman -S shellcheck  # apt-get install shellcheck 

      This list may be of particular interest to you as it describes code samples that shellcheck can protect you against. To run, simply execute shellcheck my-script, it will read the shebang (#!) directive to decide whether to analyse as shbash or ksh.

      shellcheck

      As shown in the example output above, there is one syntax error, it’s marked in red: spaces surrounding the assignment. However, shellcheck goes beyond that. In yellow markings is a warning of a potentially catastrophic event: what if toDelete is ever empty? Granted, the example is somewhat contrived, without –no-preserve-root there will be no damage should the statement resolve to /, but my point still stands! Finally, in green you will find general warnings on potential future mishaps that are best not to be ignored.

      As a closing remark on shellcheck: use the neat Syntastic Vim plugin to integrate the tool inside Vim itself, configuring when it should run, how it should behave, etc. The same script that generated the ouput above now looks like the following, from our favorite text editor.

      shellcheck-vim

      multitail

      For the sys-admins and dev-opses amongst us: stop using tail -f, and start using, multitail, tail on steroids! This ultimate log-viewer allows you to do a couple of really cool things that make it worth mentioning. Alternatively you could always either use tmux or screen to get in-shell multiplexing, or just use a modern terminal emulator allowing you to do the splitting like that, such as iTerm2 or Terminator, however in these cases you may miss some of the features below.

      • Show multiple windows at the same time, in the same shell using ncurses.
      • Merge multiple logs in the same window, for example the stdout and the stderr log of the same application.
      • Perform filtering dynamically via in-menu editable regexes.
      • Use the predefined color highlighters to make the logs more legible, or define your own, again with regular expressions.

      You know the drill how to install:

       $ brew install multitail  # pacman -S multitail  # apt-get install multitail 

      For an actual example, here is a screenshot of a window where I’m developing JRebel by attaching it to Tomcat, running the infamous petclinic project. At that time I needed both Tomcat’s own output and the JRebel one. Take note of the -CS flag, signifying “use this color scheme for all subsequent files”, followed by the name of the scheme.
      As both logs color nicely with the Apache rules, they are set, finally followed by the actual paths of the logfiles.

       $ multitail -CS apache "$TOMCAT_HOME/logs/catalina.out" "$HOME/.jrebel/jrebel.log" 

      multitail

      tree

      Our final tool in this post is the essential tree utility. Tree prints a nice, structured, tree-view of your directories, allowing you to instantly get an idea about the structure of your data, without having to lscd or z all over the place. If it is not yet pre-installed in your favourite *NIX, then grab it via your favourite package manager.

       $ brew install tree  # pacman -S tree  # apt-get install tree 

      In its most basic form, you simply type $ tree to print the current folder’s structure.

      tree

      Personally, I prefer to spice it up just a little bit, adding some flags to print human readable filesizes, hidden files, and a nice total sizecount: $ tree -ah --du.

      tree-fancy

      And that is about all I have to tell you about tree. It’s really convenient to grep its output, it gives you an awesome representation of the filesystem, and I’m sure you’ll love it from the first moment.

      That concludes my list of command line tools that you won’t think will change you live, but sure enough after some time you won’t imagine yourself not using them. In fact, you’ll likely curse every time you use a command-line-muggle’s computer without these great tools installed. What are the hidden gems of your command-line-fu? Share your favorite command line utilities in the comments below, I really would like to learn new tricks.

      posted @ 2016-04-06 15:08 小馬歌 閱讀(285) | 評(píng)論 (0)編輯 收藏
      僅列出標(biāo)題
      共95頁(yè): First 上一頁(yè) 4 5 6 7 8 9 10 11 12 下一頁(yè) Last 
       
      主站蜘蛛池模板: 国产AV无码专区亚洲AV琪琪| 亚洲国产精品热久久| 亚洲欧美国产欧美色欲| 国产精品成人免费福利| 亚洲精品视频在线观看免费| 69av免费观看| 亚洲精品国产国语| 日韩一级在线播放免费观看| 亚洲乱码中文字幕在线| 无码欧精品亚洲日韩一区夜夜嗨| 国产尤物在线视精品在亚洲| 亚洲福利在线播放| 99在线免费视频| 666精品国产精品亚洲| 99国产精品永久免费视频| 亚洲人成色4444在线观看| 国产亚洲福利一区二区免费看| 午夜免费国产体验区免费的| 国产亚洲精品自在久久 | 免费大片av手机看片| www.亚洲色图| 免费福利在线视频| 亚洲小视频在线播放| 国产禁女女网站免费看| 久久九九久精品国产免费直播| 亚洲一区二区影院| 成人免费无码大片A毛片抽搐色欲| 国产av无码专区亚洲av毛片搜| 激情97综合亚洲色婷婷五| 最近新韩国日本免费观看| 亚洲日韩精品无码专区加勒比☆| 亚洲成a人片在线观看老师| 久久精品乱子伦免费| 亚洲午夜精品久久久久久app | 成人免费网站在线观看| 一个人晚上在线观看的免费视频 | 亚洲高清在线视频| 我要看免费的毛片| 99精品全国免费观看视频..| 亚洲一级黄色大片| 国产av无码专区亚洲av果冻传媒|