<rt id="bn8ez"></rt>
<label id="bn8ez"></label>

  • <span id="bn8ez"></span>

    <label id="bn8ez"><meter id="bn8ez"></meter></label>

    石建 | Fat Mind

    2010年12月18日


    http://incubator.apache.org/kafka/design.html

    1.Why we built this
        asd(activity stream data)數(shù)據(jù)是任何網(wǎng)站的一部分,反映網(wǎng)站使用情況,如:那些內(nèi)容被搜索、展示。通常,此部分?jǐn)?shù)據(jù)被以log方式記錄在文件,然后定期的整合和分析。od(operation data)是關(guān)于機(jī)器性能數(shù)據(jù),和其它不同途徑整合的操作數(shù)據(jù)。
        在近幾年,asd和od變成一個(gè)網(wǎng)站重要的一部分,更復(fù)雜的基礎(chǔ)設(shè)施是必須的。
         數(shù)據(jù)特點(diǎn):
            a、大吞吐量的不變的ad,對(duì)實(shí)時(shí)計(jì)算是一個(gè)挑戰(zhàn),會(huì)很容易超過10倍or100倍。
     
            b、傳統(tǒng)的記錄log方式是respectable and scalable方式去支持離線處理,但是延遲太高。
        Kafka is intended to be a single queuing platform that can support both offline and online use cases.

    2.Major Design Elements

    There is a small number of major design decisions that make Kafka different from most other messaging systems:

    1. Kafka is designed for persistent messages as the common case;消息持久
    2. Throughput rather than features are the primary design constraint;吞吐量是第一要求
    3. State about what has been consumed is maintained as part of the consumer not the server;狀態(tài)由客戶端維護(hù)
    4. Kafka is explicitly distributed. It is assumed that producers, brokers, and consumers are all spread over multiple machines;必須是分布式
    3.Basics
        Messages are the fundamental unit of communication;
        Messages are
     published to a topic by a producer which means they are physically sent to a server acting as a broker,消息被生產(chǎn)者發(fā)布到一個(gè)topic,意味著物理的發(fā)送消息到broker;
        多個(gè)consumer訂閱一個(gè)topic,則此topic的每個(gè)消息都會(huì)被分發(fā)到每個(gè)consumer;
        kafka是分布式:producer、broker、consumer,均可以由集群的多臺(tái)機(jī)器組成,相互協(xié)作 a logic group;
        屬于同一個(gè)consumer group的每一個(gè)consumer process,每個(gè)消息能準(zhǔn)確的由其中的一個(gè)process消費(fèi);A more common case in our own usage is that we have multiple logical consumer groups, each consisting of a cluster of consuming machines that act as a logical whole.
        kafka不管一個(gè)topic有多少個(gè)consumer,其消息僅會(huì)存儲(chǔ)一份。

    4.Message Persistence and Caching

    4.1 Don't fear the filesystem !
        kafka完全依賴文件系統(tǒng)去存儲(chǔ)和cache消息;
        大家通常對(duì)磁盤的直覺是'很慢',則使人們對(duì)持久化結(jié)構(gòu),是否能提供有競爭力的性能表示懷疑;實(shí)際上,磁盤到底有多慢或多塊,完全取決于如何使用磁盤,a properly designed disk structure can often be as fast as the network.
        http://baike.baidu.com/view/969385.htm raid-5 
        http://www.china001.com/show_hdr.php?xname=PPDDMV0&dname=66IP341&xpos=172 磁盤種類
        磁盤順序讀寫的性能非常高, linear writes on a 6 7200rpm SATA RAID-5 array is about 300MB/sec;These linear reads and writes are the most predictable of all usage patterns, and hence the one detected and optimized best by the operating system using read-ahead and write-behind techniques。順序讀寫是最可預(yù)見的模式,因此操作系統(tǒng)通過read-head和write-behind技術(shù)去優(yōu)化。
        現(xiàn)代操作系統(tǒng),用mem作為disk的cache;Any modern OS will happily divert all free memory to disk caching with little performance penalty when the memory is reclaimed. All disk reads and writes will go through this unified cache. 
        Jvm:a、對(duì)象的內(nèi)存開銷是非常大的,通常是數(shù)據(jù)存儲(chǔ)的2倍;b、當(dāng)heap數(shù)據(jù)增大時(shí),gc代價(jià)越來越大;
        As a result of these factors using the filesystem and relying on pagecache is superior to maintaining an in-memory cache or other structure。依賴文件系統(tǒng)和pagecache是優(yōu)于mem cahce或其它結(jié)構(gòu)的。
        數(shù)據(jù)壓縮,Doing so will result in a cache of up to 28-30GB on a 32GB machine without GC penalties. 
        This suggests a design which is very simple: maintain as much as possible in-memory and flush to the filesystem only when necessary. 盡可能的維持在內(nèi)存中,僅當(dāng)必須時(shí)寫回到文件系統(tǒng).
        當(dāng)數(shù)據(jù)被立即寫回到持久化的文件,而未調(diào)用flush,其意味著數(shù)據(jù)僅被寫入到os pagecahe,在后續(xù)某個(gè)時(shí)間由os flush。Then we add a configuration driven flush policy to allow the user of the system to control how often data is flushed to the physical disk (every N messages or every M seconds) to put a bound on the amount of data "at risk" in the event of a hard crash. 提供flush策略。

    4.2 
    Constant Time Suffices
        
    The persistent data structure used in messaging systems metadata is often a BTree. BTrees are the most versatile data structure available, and make it possible to support a wide variety of transactional and non-transactional semantics in the messaging system.
        Disk seeks come at 10 ms a pop, and each disk can do only one seek at a time so parallelism is limited. Hence even a handful of disk seeks leads to very high overhead. 
        Furthermore BTrees require a very sophisticated page or row locking implementation to avoid locking the entire tree on each operation.
    The implementation must pay a fairly high price for row-locking or else effectively serialize all reads.
        持久化消息的元數(shù)據(jù)通常是BTree結(jié)構(gòu),但磁盤結(jié)構(gòu),其代價(jià)太大。原因:尋道、避免鎖整棵樹。
        
    Intuitively a persistent queue could be built on simple reads and appends to files as is commonly the case with logging solutions.
        持久化隊(duì)列可以構(gòu)建在讀和append to 文件。所以不支持BTree的一些語義,但其好處是:O(1)消耗,無鎖讀寫。
        
    the performance is completely decoupled from the data size--one server can now take full advantage of a number of cheap, low-rotational speed 1+TB SATA drives. 
    Though they have poor seek performance, these drives often have comparable performance for large reads and writes at 1/3 the price and 3x the capacity.

    4.3 Maximizing Efficiency
        Furthermore we assume each message published is read at least once (and often multiple times), hence we optimize for consumption rather than production. 更進(jìn)一步,我們假設(shè)被發(fā)布的消息至少會(huì)讀一次,因此優(yōu)化consumer優(yōu)先于producer。
        
    There are two common causes of inefficiency :
            two many network requests, (
     APIs are built around a "message set" abstraction,
    This allows network requests to group messages together and amortize the overhead of the network roundtrip rather than sending a single message at a time.) 僅提供批量操作api,則每次網(wǎng)絡(luò)開銷是平分在一組消息,而不是單個(gè)消息。
        and excessive byte copying.(
    The message log maintained by the broker is itself just a directory of message sets that have been written to disk.
    Maintaining this common format allows optimization of the most important operation : network transfer of persistent log chunks.
        To understand the impact of sendfile, it is important to understand the common data path for transfer of data from file to socket:
    1. The operating system reads data from the disk into pagecache in kernel space
    2. The application reads the data from kernel space into a user-space buffer
    3. The application writes the data back into kernel space into a socket buffer
    4. The operating system copies the data from the socket buffer to the NIC buffer where it is sent over the network
        利用os提供的zero-copy,
    only the final copy to the NIC buffer is needed.

    4.4 End-to-end Batch Compression
        In many cases the bottleneck is actually not CPU but network. This is particularly true for a data pipeline that needs to send messages across data centers.
    Efficient compression requires compressing multiple messages together rather than compressing each message individually. 
    Ideally this would be possible in an end-to-end fashion — that is, data would be compressed prior to sending by the producer and remain compressed on the server, only being decompressed by the eventual consumers. 
        
    A batch of messages can be clumped together compressed and sent to the server in this form. This batch of messages will be delivered all to the same consumer and will remain in compressed form until it arrives there.
        理解:kafka 
    producer api 提供批量壓縮,broker不對(duì)此批消息做任何操作,且以壓縮的方式,一起被發(fā)送到consumer。

    4.5 Consumer state
        Keeping track of what has been consumed is one of the key things a messaging system must provide. 
    State tracking requires updating a persistent entity and potentially causes random accesses. 
        
    Most messaging systems keep metadata about what messages have been consumed on the broker. That is, as a message is handed out to a consumer, the broker records that fact locally. 大部分消息系統(tǒng),存儲(chǔ)是否被消費(fèi)的元信息在broker。則是說,一個(gè)消息被分發(fā)到一個(gè)consumer,broker記錄。
        問題:當(dāng)consumer消費(fèi)失敗后,會(huì)導(dǎo)致消息丟失;改進(jìn):每次consumer消費(fèi)后,給broker ack,若broker在超時(shí)時(shí)間未收到ack,則重發(fā)此消息。
        問題:1.當(dāng)消費(fèi)成功,但未ack時(shí),會(huì)導(dǎo)致消費(fèi)2次  2.
     now the broker must keep multiple states about every single message  3.當(dāng)broker是多臺(tái)機(jī)器時(shí),則狀態(tài)之間需要同步

    4.5.1 Message delivery semantics
        
    So clearly there are multiple possible message delivery guarantees that could be provided : at most once 、at least once、exactly once。
        
    This problem is heavily studied, and is a variation of the "transaction commit" problem. Algorithms that provide exactly once semantics exist, two- or three-phase commits and Paxos variants being examples, but they come with some drawbacks. They typically require multiple round trips and may have poor guarantees of liveness (they can halt indefinitely). 
        消費(fèi)分發(fā)語義,是 ‘事務(wù)提交’ 問題的變種。算法提供 exactly onece 語義,兩階段 or 三階段提交,paxos 均是例子,但它們存在缺點(diǎn)。典型的問題是要求多次round trip,且
    poor guarantees of liveness。
        
    Kafka does two unusual things with respect to metadata. 
    First the stream is partitioned on the brokers into a set of distinct partitions. 
    Within a partition messages are stored in the order in which they arrive at the broker, and will be given out to consumers in that same order. This means that rather than store metadata for each message (marking it as consumed, say), we just need to store the "high water mark" for each combination of consumer, topic, and partition.  
        
    4.5.2 
    Consumer state
        In Kafka, the consumers are responsible for maintaining state information (offset) on what has been consumed. 
    Typically, the Kafka consumer library writes their state data to zookeeper.
        
    This solves a distributed consensus problem, by removing the distributed part!
        
    There is a side benefit of this decision. A consumer can deliberately rewind back to an old offset and re-consume data.

    4.5.3 Push vs. pull
        
    A related question is whether consumers should pull data from brokers or brokers should push data to the subscriber.
    There are pros and cons to both approaches.
        However a push-based system has difficulty dealing with diverse consumers as the broker controls the rate at which data is transferred. push目標(biāo)是consumer能在最大速率去消費(fèi),可不幸的是,當(dāng)consume速率小于生產(chǎn)速率時(shí),the consumer tends to be overwhelmed。
        
    A pull-based system has the nicer property that the consumer simply falls behind and catches up when it can. This can be mitigated with some kind of backoff protocol by which the consumer can indicate it is overwhelmed, but getting the rate of transfer to fully utilize (but never over-utilize) the consumer is trickier than it seems. Previous attempts at building systems in this fashion led us to go with a more traditional pull model.  不存在push問題,且也保證充分利用consumer能力。

    5. Distribution
        Kafka is built to be run across a cluster of machines as the common case. There is no central "master" node. Brokers are peers to each other and can be added and removed at anytime without any manual configuration changes. Similarly, producers and consumers can be started dynamically at any time. Each broker registers some metadata (e.g., available topics) in Zookeeper. Producers and consumers can use Zookeeper to discover topics and to co-ordinate the production and consumption. The details of producers and consumers will be described below.

    6. Producer

    6.1 Automatic producer load balancing
        Kafka supports client-side load balancing for message producers or use of a dedicated load balancer to balance TCP connections. 
     
        The advantage of using a level-4 load balancer is that each producer only needs a single TCP connection, and no connection to zookeeper is needed. 
    The disadvantage is that the balancing is done at the TCP connection level, and hence it may not be well balanced (if some producers produce many more messages then others, evenly dividing up the connections per broker may not result in evenly dividing up the messages per broker).
        
    Client-side zookeeper-based load balancing solves some of these problems. It allows the producer to dynamically discover new brokers, and balance load on a per-request basis. It allows the producer to partition data according to some key instead of randomly.

        The working of the zookeeper-based load balancing is described below. Zookeeper watchers are registered on the following events—

    • a new broker comes up
    • a broker goes down
    • a new topic is registered
    • a broker gets registered for an existing topic

        Internally, the producer maintains an elastic pool of connections to the brokers, one per broker. This pool is kept updated to establish/maintain connections to all the live brokers, through the zookeeper watcher callbacks. When a producer request for a particular topic comes in, a broker partition is picked by the partitioner (see section on semantic partitioning). The available producer connection is used from the pool to send the data to the selected broker partition.
        producer通過zk,管理與broker的連接。當(dāng)一個(gè)請(qǐng)求,根據(jù)partition rule 計(jì)算分區(qū),從連接池選擇對(duì)應(yīng)的connection,發(fā)送數(shù)據(jù)。

    6.2 Asynchronous send

        Asynchronous non-blocking operations are fundamental to scaling messaging systems.
        
    This allows buffering of produce requests in a in-memory queue and batch sends that are triggered by a time interval or a pre-configured batch size. 

    6.3 Semantic partitioning
        
    The producer has the capability to be able to semantically map messages to the available kafka nodes and partitions. 
    This allows partitioning the stream of messages with some semantic partition function based on some key in the message to spread them over broker machines. 


    posted @ 2013-07-06 14:57 石建 | Fat Mind 閱讀(1761) | 評(píng)論 (0)編輯 收藏

    1.Js代碼,login.js文件

    //用戶的登陸信息寫入cookies
    function SetCookie(form)//兩個(gè)參數(shù),一個(gè)是cookie的名子,一個(gè)是值
    {   
        var name = form.name.value;
        var password = form.password.value;
        var Days = 1; //此 cookie 將被保存 7 天 
        var exp  = new Date(); //生成一個(gè)現(xiàn)在的日期,加上保存期限,然后設(shè)置cookie的生存期限!
        exp.setTime(exp.getTime() + Days*24*60*60*1000);
        document.cookie = "user="+ escape(name) + "/" + escape(password) + ";expires=" + exp.toGMTString();
    }
    //取cookies函數(shù)--正則表達(dá)式(不會(huì),學(xué)習(xí)正則表達(dá)式)  
    function getCookie(name)      
    {
        var arr = document.cookie.match(new RegExp("(^| )"+name+"=([^;]*)(;|$)"));
        if(arr != nullreturn unescape(arr[2]); 
        return null;
    }
    //取cookies函數(shù)--普通實(shí)現(xiàn)      
      function   readCookie(form){   
          var   cookieValue   =   "";   
          var   search   =   "user=";   
          if(document.cookie.length   >   0)     {   
              offset   =   document.cookie.indexOf(search);
              if(offset !=  -1){     
                  offset   +=   search.length;   
                  end   =   document.cookie.indexOf(";",offset);   
                  if   (end  ==  -1)   
                        end   =   document.cookie.length;
                  //獲取cookies里面的值          
                  cookieValue   =   unescape(document.cookie.substring(offset,end))
                  if(cookieValue != null){
                        var str = cookieValue.split("/");
                        form.name.value = str[0];
                        form.password.value = str[1]; 
                  }
              }   
          }    
      }   
    //刪除cookie,(servlet里面:設(shè)置時(shí)間為0,設(shè)置為-1和session的范圍是一樣的),javascript好像是有點(diǎn)區(qū)別
    function delCookie()
    {
        var name = "admin";
        var exp = new Date();
        exp.setTime(exp.getTime() - 1);
        var cval=getCookie(name);
        if(cval!=null) document.cookie= name + "="+cval+";expires="+exp.toGMTString();
    }

     

    2.jsp代碼,文件login.jsp

    <%@ page contentType="text/html; charset=gb2312" language="java"
        import="java.sql.*" errorPage=""%>
        
    <html>
        <head>
            <meta http-equiv="Content-Type" content="text/html; charset=gb2312">
            <title>javascript 控制 cookie</title>
            <link href="css/style.css" rel="stylesheet" type="text/css">
            <script type="text/javascript" src="js/login.js"></script>
        </head>
        <script language="javascript">
        function checkEmpty(form){
            for(i=0;i<form.length;i++){
                if(form.elements[i].value==""){
                    alert("表單信息不能為空");
                    return false;
                }
            }
        }
    </script>
        <body onload="readCookie(form)"> <!-- 作為JavaScript控制的cookie-->
            <div align="center">
                <table width="324" height="225" border="0" cellpadding="0" cellspacing="0">
                    <tr height="50">
                        <td ></td>
                    </tr>
                    <tr align="center">
                        <td background="images/back.jpg">
                            <br>
                            <br>
                            登陸
                            <form name="form" method="post" action="" onSubmit="return checkEmpty(form)">
                                <input type="hidden" name="id" value="-1">
                                <table width="268" border="1" cellpadding="0" cellspacing="0">
                                    <tr align="center">
                                        <td width="63" height="30">
                                            用戶名:
                                        </td>
                                        <td width="199">
                                            <input type="text" name="name" id="name">
                                        </td>
                                    </tr>
                                    <tr align="center">
                                        <td height="30">
                                            密碼:
                                        </td>
                                        <td>
                                            <input type="password" name="password" id="password">
                                        </td>
                                    </tr>
                                </table>
                                <br>
                                <input type="submit" value="提交">
                                <input type="checkbox" name="cookie" onclick="SetCookie(form)">記住我          
                            </form>
                        </td>
                    </tr>
                </table>
            </div>
        </body>
    </html>

     


    目的:當(dāng)你再次打開login.jsp頁面,表單里面的內(nèi)容已經(jīng)寫好了,是你上一次的登陸信息!


    問題:1.JavaScript里面取cookie都是寫死的,不是很靈活!
                2.JavaScript的cookie是按照字符串的形式存放的,所以拿出的時(shí)候,你要按照你放進(jìn)去的形式來選擇!
                3.本來是想實(shí)現(xiàn)自動(dòng)登陸的,可我的每個(gè)頁面都要session的檢查!一個(gè)客戶端,一個(gè)服務(wù)器端,沒能實(shí)現(xiàn)!

     

     

    posted @ 2012-09-09 15:18 石建 | Fat Mind 閱讀(638) | 評(píng)論 (0)編輯 收藏
    1.變量類型
      - undefined
      - null
      - string
            - == 與 === 區(qū)別
      - number
      - boolean
      - string、number、boolean均有對(duì)應(yīng)的 '對(duì)象類'
    2.函數(shù)
      - 定義函數(shù)
            - function 關(guān)鍵字
            - 參數(shù)(見例子),arguments
            - 函數(shù)內(nèi)變量聲明,var區(qū)別
      - 作用域
            - 鏈?zhǔn)浇Y(jié)構(gòu)(子函數(shù)可以看見父函數(shù)的變量)
      - 匿名函數(shù)
          - 使用場景(非復(fù)用場景,如:jsonp回調(diào)函數(shù))
          - this特征
    例子:
    var add = function(x) {
        return x++;

    }
    add(1,2,3); // 參數(shù)可以隨意多個(gè),類似Java中的(int x ...)

    var fn = function(name, pass) {
        alert(name);
        alert(pass);
    };
    fn("hello","1234",5); // 按照傳遞的順序排列


    var name = "windows";
    var fn = function() {
        var name 
    = "hello";
        alert(
    this.name);
    }
    fn(); // windows,this在匿名函數(shù)內(nèi)部是指向windows范圍

    var name = "windows";
    var fn = function() {
        name 
    = "hello";
        alert(
    this.name);
    }
    fn(); // 因函數(shù)內(nèi)部變量name未聲明為var,則屬于全局變量,且this指向windows,則為'hello'

    function add(a) {
        return ++a;
    }
    var fn = function(x,add){
        return add(x);
    }
    fn(1, add);  // 函數(shù)作為參數(shù)

    3.閉包  
    http://www.ruanyifeng.com/blog/2009/08/learning_javascript_closures.html 【good】
    其它語言閉包概念 http://www.ibm.com/developerworks/cn/linux/l-cn-closure/

    4.對(duì)象
        - new Object()
        – 對(duì)象字面量
        – 構(gòu)造函數(shù)
        - 上述操作,經(jīng)歷的步驟
            –創(chuàng)建新對(duì)象
            –將構(gòu)造方法的作用域賦給新對(duì)象(new 操作符)
            –為對(duì)象添加屬性, 方法
            –返回該對(duì)象

    var obj = new Object();  // new Object方式
    obj.name = 'zhangsan';

    var obj = {                   // 字面常量方式,定義對(duì)象
        name : 'zhangsan',
        showName : function (){
            alert(this.name);
        }
    };
    alert(obj.showName());
    function Person(name) { // 構(gòu)造函數(shù)
        this.name = name;
        this.showName = function(){
            return this.name; }
        };
    var obj = new Person("zhangsan"); // 必須用 new 關(guān)鍵,否則等于調(diào)用一個(gè)普通函數(shù)
    obj.showName();
    alert(obj.name);


    資料:內(nèi)部培訓(xùn)ppt 
    posted @ 2012-05-20 13:50 石建 | Fat Mind 閱讀(263) | 評(píng)論 (0)編輯 收藏


    1.句柄就是一個(gè)標(biāo)識(shí)符,只要獲得對(duì)象的句柄,我們就可以對(duì)對(duì)象進(jìn)行任意的操作。

    2.句柄不是指針,操作系統(tǒng)用句柄可以找到一塊內(nèi)存,這個(gè)句柄可能是標(biāo)識(shí)符,mapkey,也可能是指針,看操作系統(tǒng)怎么處理的了。

    fd算是在某種程度上替代句柄吧;

    Linux 有相應(yīng)機(jī)制,但沒有統(tǒng)一的句柄類型,各種類型的系統(tǒng)資源由各自的類型來標(biāo)識(shí),由各自的接口操作。

    3.http://tech.ddvip.com/2009-06/1244006580122204_11.html

    在操作系統(tǒng)層面上,文件操作也有類似于FILE的一個(gè)概念,在Linux里,這叫做文件描述符(File Descriptor),而在Windows里,叫做句柄(Handle)(以下在沒有歧義的時(shí)候統(tǒng)稱為句柄)。用戶通過某個(gè)函數(shù)打開文件以獲得句柄,此 后用戶操縱文件皆通過該句柄進(jìn)行。

     

    設(shè)計(jì)這么一個(gè)句柄的原因在于句柄可以防止用戶隨意讀寫操作系統(tǒng)內(nèi)核的文件對(duì)象。無論是Linux還是Windows,文件句柄總是和內(nèi)核的文件對(duì)象相關(guān)聯(lián)的,但如何關(guān)聯(lián)細(xì)節(jié)用戶并不可見。內(nèi)核可以通過句柄來計(jì)算出內(nèi)核里文件對(duì)象的地址,但此能力并不對(duì)用戶開放。

     

    下面舉一個(gè)實(shí)際的例子,在Linux中,值為012fd分別代表標(biāo)準(zhǔn)輸入、標(biāo)準(zhǔn)輸出和標(biāo)準(zhǔn)錯(cuò)誤輸出。在程序中打開文件得到的fd3開始增長。 fd具體是什么呢?在內(nèi)核中,每一個(gè)進(jìn)程都有一個(gè)私有的打開文件表,這個(gè)表是一個(gè)指針數(shù)組,每一個(gè)元素都指向一個(gè)內(nèi)核的打開文件對(duì)象。而fd,就是這 個(gè)表的下標(biāo)。當(dāng)用戶打開一個(gè)文件時(shí),內(nèi)核會(huì)在內(nèi)部生成一個(gè)打開文件對(duì)象,并在這個(gè)表里找到一個(gè)空項(xiàng),讓這一項(xiàng)指向生成的打開文件對(duì)象,并返回這一項(xiàng)的下標(biāo) 作為fd。由于這個(gè)表處于內(nèi)核,并且用戶無法訪問到,因此用戶即使擁有fd,也無法得到打開文件對(duì)象的地址,只能夠通過系統(tǒng)提供的函數(shù)來操作。

     

    C語言里,操縱文件的渠道則是FILE結(jié)構(gòu),不難想象,C語言中的FILE結(jié)構(gòu)必定和fd有一對(duì)一的關(guān)系,每個(gè)FILE結(jié)構(gòu)都會(huì)記錄自己唯一對(duì)應(yīng)的fd。


    句柄 
    http://zh.wikipedia.org/wiki/%E5%8F%A5%E6%9F%84

    程序設(shè)計(jì) ,句柄是一種特殊的智能指針 。當(dāng)一個(gè)應(yīng)用程序 要引用其他系統(tǒng)(數(shù)據(jù)庫、操作系統(tǒng) )所管理的內(nèi)存 塊或對(duì)象 時(shí),就要使用句柄。

    句柄與普通指針 的區(qū)別在于,指針包含的是引用對(duì)象 內(nèi)存地址 ,而句柄則是由系統(tǒng)所管理的引用標(biāo)識(shí),該標(biāo)識(shí)可以被系統(tǒng)重新定位到一個(gè)內(nèi)存地址 上。這種間接訪問對(duì)象 的模式增強(qiáng)了系統(tǒng)對(duì)引用對(duì)象 的控制。(參見封裝 )。

    在上世紀(jì)80年代的操作系統(tǒng)(如Mac OS Windows )的內(nèi)存管理 中,句柄被廣泛應(yīng)用。Unix 系統(tǒng)的文件描述符 基本上也屬于句柄。和其它桌面環(huán)境 一樣,Windows API 大量使用句柄來標(biāo)識(shí)系統(tǒng)中的對(duì)象 ,并建立操作系統(tǒng)與用戶空間 之間的通信渠道。例如,桌面上的一個(gè)窗體由一個(gè)HWND 類型的句柄來標(biāo)識(shí)。

    如今,內(nèi)存 容量的增大和虛擬內(nèi)存 算法使得更簡單的指針 愈加受到青睞,而指向另一指針的那類句柄受到冷淡。盡管如此,許多操作系統(tǒng) 仍然把指向私有對(duì)象 的指針以及進(jìn)程傳遞給客戶端 的內(nèi)部數(shù)組 下標(biāo)稱為句柄。


     

    posted @ 2012-04-06 14:02 石建 | Fat Mind 閱讀(11889) | 評(píng)論 (0)編輯 收藏

    官方 :http://code.google.com/p/powermock/ 

    1. 使用mockito的同學(xué),推薦閱讀如下部分

        - document [必選]
            - getting started
            - motavition
        - mockito extends [必選]
            - mockito 1.8+ useage
        - common
        - tutorial
        - faq [必選]


    2. 附件:實(shí)際開發(fā)中使用到的
    powermock的一些特性,簡化后的例子(僅為說明powermock api使用)。主要包括

     

    -          修改私有域

    -          私有方法

    -            測試私有方法

    -            Mock

    -            Verify

    -          靜態(tài)方法

    -            Mock

    -            拋出異常

    -            Verify

    -          Mock類部分方法

    -          Mock Java core library,如:Thread

    -          Mock 構(gòu)造器

    /Files/shijian/powermock.rar



    posted @ 2012-03-29 12:39 石建 | Fat Mind 閱讀(902) | 評(píng)論 (0)編輯 收藏


    原文地址:
    http://googleresearch.blogspot.com/2012/03/excellent-papers-for-2011.html

     

    Excellent Papers for 2011

    Posted by Corinna Cortes and Alfred Spector, Google Research

    Googlers across the company actively engage with the scientific community by publishing technical papers, contributing open-source packages, working on standards, introducing new APIs and tools, giving talks and presentations, participating in ongoing technical debates, and much more. Our publications offer technical and algorithmic advances, feature aspects we learn as we develop novel products and services, and shed light on some of the technical challenges we face at Google.

     

    谷歌公司積極參與科學(xué)界的交流,通過發(fā)表技術(shù)論文,貢獻(xiàn)開源軟件,制定標(biāo)準(zhǔn),引入新的API和工具,舉辦講座和演講,參加正在進(jìn)行的技術(shù)辯論,等等。我們發(fā)布的文章提供技術(shù)和算法的進(jìn)步,在開發(fā)新的產(chǎn)品和服務(wù)過程中學(xué)習(xí)到的內(nèi)容,揭示一些我們?cè)诠雀杷媾R的技術(shù)挑戰(zhàn)。

     

    In an effort to highlight some of our work, we periodically select a number of publications to be featured on this blog. We first posted a set of papers on this blog in mid-2010 and subsequently discussed them in more detail in the following blog postings. In a second round, we highlighted new noteworthy papers from the later half of 2010. This time we honor the influential papers authored or co-authored by Googlers covering all of 2011 -- covering roughly 10% of our total publications.  It’s tough choosing, so we may have left out some important papers.  So, do see the publications list to review the complete group.

     

    為了彰顯我們的一些工作,我們定期選擇一些列文章發(fā)布在blog。2010中期,我們第一次發(fā)布了一些列的文章在blog,并隨后在博客文章中更詳細(xì)討論它們。在第二輪中,我們強(qiáng)調(diào)從2010年下半年新值得注意的論文。這一次,我們給有影響力的文章的作者或合著者以榮譽(yù),大約占總文章數(shù)的10%。這是艱難的選擇的,所以我們可能已經(jīng)遺漏了一些重要文章。因此,請(qǐng)看完整的文章清單。

     

    In the coming weeks we will be offering a more in-depth look at these publications, but here are some summaries:

     

    在未來幾周我們將更深入的談?wù)撨@些論文,但現(xiàn)在只做一些總結(jié)。

     

    Audio processing

     

    Cascades of two-pole–two-zero asymmetric resonators are good models of peripheral auditory function”, Richard F. Lyon,Journal of the Acoustical Society of America, vol. 130 (2011), pp. 3893-3904.
    Lyon's long title summarizes a result that he has been working toward over many years of modeling sound processing in the inner ear. 
     This nonlinear cochlear model is shown to be "good" with respect to psychophysical data on masking, physiological data on mechanical and neural response, and computational efficiency. These properties derive from the close connection between wave propagation and filter cascades. This filter-cascade model of the ear is used as an efficient sound processor for several machine hearing projects at Google.

     

    聲音處理:這個(gè)濾波器級(jí)聯(lián)模型的耳朵是用來作為一種高效的聲音處理器,是谷歌的幾個(gè)機(jī)器聲音處理項(xiàng)目之一。

     

    Electronic Commerce and Algorithms

     

    Online Vertex-Weighted Bipartite Matching and Single-bid Budgeted Allocations”, Gagan AggarwalGagan GoelChinmay KarandeAranyak MehtaSODA 2011.
    The authors introduce an elegant and powerful algorithmic technique to the area of online ad allocation and matching: a hybrid of random perturbations and greedy choice to make decisions on the fly. Their technique sheds new light on classic matching algorithms, and can be used, for example, to pick one among a set of relevant ads, without knowing in advance the demand for ad slots on future web page views. 

     

    作者介紹在線廣告分配和匹配方面的優(yōu)雅和強(qiáng)大的算法技術(shù):一種混合隨機(jī)擾動(dòng)和貪婪選擇,實(shí)現(xiàn)在線決定。他們的技術(shù)揭示了經(jīng)典的匹配算法的新的方向,例如,挑選其中一組相關(guān)的廣告,事先不知道未來的網(wǎng)站頁面訪問的廣告位置的需求?!娟P(guān)注】

     

    Milgram-routing in social networks”, Silvio Lattanzi, Alessandro Panconesi, D. Sivakumar, Proceedings of the 20th International Conference on World Wide Web, WWW 2011, pp. 725-734.
    Milgram’s "six-degrees-of-separation experiment" and the fascinating small world hypothesis that follows from it, have generated a lot of interesting research in recent years. In this landmark experiment, Milgram showed that people unknown to each other are often connected by surprisingly short chains of acquaintances. In the paper we prove theoretically and experimentally how a recent model of social networks, "Affiliation Networks", offers an explanation to this phenomena and inspires interesting technique for local routing within social networks.

     

    米爾格蘭姆的六個(gè)度分離實(shí)驗(yàn),迷人的小世界遵從它的結(jié)果,在最近幾年已經(jīng)產(chǎn)生了很多有趣的研究。在這一具有里程碑意義的實(shí)驗(yàn),表明未知的對(duì)方往往是通過熟人,以令人驚訝的短鏈連接即可認(rèn)識(shí)。在本文中,我們提供理論和實(shí)驗(yàn)關(guān)于近代的社會(huì)網(wǎng)絡(luò)模型,Affiliation Networks,提供了一種解釋這種現(xiàn)象,并激發(fā)社會(huì)網(wǎng)絡(luò)的interesting technique for local routing?!娟P(guān)注】

     

    Non-Price Equilibria in Markets of Discrete Goods”, Avinatan Hassidim, Haim Kaplan, Yishay Mansour, Noam Nisan, EC, 2011.
    We present a correspondence between markets of indivisible items, and a family of auction based n player games. We show that a market has a price based (Walrasian) equilibrium if and only if the corresponding game has a pure Nash equilibrium. We then turn to markets which do not have a Walrasian equilibrium (which is the interesting case), and study properties of the mixed Nash equilibria of the corresponding games.

     

    在離散商品市場的非價(jià)格平衡【關(guān)注】

     

    HCI

     

    From Basecamp to Summit: Scaling Field Research Across 9 Locations”, Jens Riegelsberger, Audrey Yang, Konstantin Samoylov, Elizabeth Nunge, Molly Stevens, Patrick Larvie, CHI 2011 Extended Abstracts.
    The paper reports on our experience with a basecamp research hub to coordinate logistics and ongoing real-time analysis with research teams in the field. We also reflect on the implications for the meaning of research in a corporate context, where much of the value may be less in a final report, but more in the curated impressions and memories our colleagues take away from the the research trip.

    User-Defined Motion Gestures for Mobile Interaction”, Jaime Ruiz, Yang Li, Edward Lank, CHI 2011: ACM Conference on Human Factors in Computing Systems, pp. 197-206.
    Modern smartphones contain sophisticated sensors that can detect rich motion gestures — deliberate movements of the device by end-users to invoke commands. However, little is known about best-practices in motion gesture design for the mobile computing paradigm. We systematically studied the design space of motion gestures via a guessability study that elicits end-user motion gestures to invoke commands on a smartphone device. The study revealed consensus among our participants on parameters of movement and on mappings of motion gestures onto commands, by which we developed a taxonomy for motion gestures and compiled an end-user inspired motion gesture set. The work lays the foundation of motion gesture design—a new dimension for mobile interaction.

    Information Retrieval

    Reputation Systems for Open Collaboration”, B.T. Adler, L. de Alfaro, A. Kulshrestra, I. Pye, Communications of the ACM, vol. 54 No. 8 (2011), pp. 81-87.
    This paper describes content based reputation algorithms, that rely on automated content analysis to derive user and content reputation, and their applications for Wikipedia and google Maps. The Wikipedia reputation system WikiTrust relies on a chronological analysis of user contributions to articles, metering positive or negative increments of reputation whenever new contributions are made. The Google Maps system Crowdsensus compares the information provided by users on map business listings and computes both a likely reconstruction of the correct listing and a reputation value for each user. Algorithmic-based user incentives ensure the trustworthiness of evaluations of Wikipedia entries and Google Maps business information.

    Machine Learning and Data Mining

    Domain adaptation in regression”, Corinna CortesMehryar MohriProceedings of The 22nd International Conference on Algorithmic Learning Theory, ALT 2011.
    Domain adaptation is one of the most important and challenging problems in machine learning. 
     This paper presents a series of theoretical guarantees for domain adaptation in regression, gives an adaptation algorithm based on that theory that can be cast as a semi-definite programming problem, derives an efficient solution for that problem by using results from smooth optimization, shows that the solution can scale to relatively large data sets, and reports extensive empirical results demonstrating the benefits of this new adaptation algorithm.

    On the necessity of irrelevant variables”, David P. Helmbold, Philip M. LongICML, 2011
    Relevant variables sometimes do much more good than irrelevant variables do harm, so that it is possible to learn a very accurate classifier using predominantly irrelevant variables. 
     We show that this holds given an assumption that formalizes the intuitive idea that the variables are non-redundant.  For problems like this it can be advantageous to add many additional variables, even if only a small fraction of them are relevant.

    Online Learning in the Manifold of Low-Rank Matrices”, Gal Chechik, Daphna Weinshall, Uri Shalit, Neural Information Processing Systems (NIPS 23), 2011, pp. 2128-2136.
    Learning measures of similarity from examples of similar and dissimilar pairs is a problem that is hard to scale. LORETA uses retractions, an operator from matrix optimization, to learn low-rank similarity matrices efficiently. This allows to learn similarities between objects like images or texts when represented using many more features than possible before.

    Machine Translation

    Training a Parser for Machine Translation Reordering”, Jason Katz-Brown, Slav PetrovRyan McDonaldFranz Och, David Talbot, Hiroshi Ichikawa, Masakazu Seno, Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP '11).
    Machine translation systems often need to understand the syntactic structure of a sentence to translate it correctly. Traditionally, syntactic parsers are evaluated as standalone systems against reference data created by linguists. Instead, we show how to train a parser to optimize reordering accuracy in a machine translation system, resulting in measurable improvements in translation quality over a more traditionally trained parser.

    Watermarking the Outputs of Structured Prediction with an application in Statistical Machine Translation”, Ashish Venugopal,Jakob Uszkoreit, David Talbot, Franz Och, Juri Ganitkevitch, Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP).
    We propose a general method to watermark and probabilistically identify the structured results of machine learning algorithms with an application in statistical machine translation. Our approach does not rely on controlling or even knowing the inputs to the algorithm and provides probabilistic guarantees on the ability to identify collections of results from one’s own algorithm, while being robust to limited editing operations.

    Inducing Sentence Structure from Parallel Corpora for Reordering”, John DeNeroJakob UszkoreitProceedings of the 2011 Conference on Empirical Methods in Natural Language Processing (EMNLP).
    Automatically discovering the full range of linguistic rules that govern the correct use of language is an appealing goal, but extremely challenging. 
     Our paper describes a targeted method for discovering only those aspects of linguistic syntax necessary to explain how two different languages differ in their word ordering.  By focusing on word order, we demonstrate an effective and practical application of unsupervised grammar induction that improves a Japanese to English machine translation system.

    Multimedia and Computer Vision

    Kernelized Structural SVM Learning for Supervised Object Segmentation”, Luca BertelliTianli Yu, Diem Vu, Burak Gokturk,Proceedings of IEEE Conference on Computer Vision and Pattern Recognition 2011.
    The paper proposes a principled way for computers to learn how to segment the foreground from the background of an image given a set of training examples. The technology is build upon a specially designed nonlinear segmentation kernel under the recently proposed structured SVM learning framework.

    Auto-Directed Video Stabilization with Robust L1 Optimal Camera Paths”, Matthias GrundmannVivek Kwatra, Irfan Essa,IEEE Conference on Computer Vision and Pattern Recognition (CVPR 2011).
    Casually shot videos captured by handheld or mobile cameras suffer from significant amount of shake. Existing in-camera stabilization methods dampen high-frequency jitter but do not suppress low-frequency movements and bounces, such as those observed in videos captured by a walking person. On the other hand, most professionally shot videos usually consist of carefully designed camera configurations, using specialized equipment such as tripods or camera dollies, and employ ease-in and ease-out for transitions. Our stabilization technique automatically converts casual shaky footage into more pleasant and professional looking videos by mimicking these cinematographic principles. The original, shaky camera path is divided into a set of segments, each approximated by either constant, linear or parabolic motion, using an algorithm based on robust L1 optimization. The stabilizer has been part of the YouTube Editor (youtube.com/editor) since March 2011.

    The Power of Comparative Reasoning”, Jay Yagnik, Dennis Strelow, David Ross, Ruei-Sung Lin, International Conference on Computer Vision (2011).
    The paper describes a theory derived vector space transform that converts vectors into sparse binary vectors such that Euclidean space operations on the sparse binary vectors imply rank space operations in the original vector space. The transform a) does not need any data-driven supervised/unsupervised learning b) can be computed from polynomial expansions of the input space in linear time (in the degree of the polynomial) and c) can be implemented in 10-lines of code. We show competitive results on similarity search and sparse coding (for classification) tasks.

    NLP

    Unsupervised Part-of-Speech Tagging with Bilingual Graph-Based Projections”, Dipanjan Das, Slav PetrovProceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL '11), 2011, Best Paper Award.
    We would like to have natural language processing systems for all languages, but obtaining labeled data for all languages and tasks is unrealistic and expensive. We present an approach which leverages existing resources in one language (for example English) to induce part-of-speech taggers for languages without any labeled training data. We use graph-based label propagation for cross-lingual knowledge transfer and use the projected labels as features in a hidden Markov model trained with the Expectation Maximization algorithm.

    Networks

    TCP Fast Open”, Sivasankar Radhakrishnan, Yuchung ChengJerry ChuArvind Jain, Barath Raghavan, Proceedings of the 7th International Conference on emerging Networking EXperiments and Technologies (CoNEXT), 2011.
    TCP Fast Open enables data exchange during TCP’s initial handshake. It decreases application network latency by one full round-trip time, a significant speedup for today's short Web transfers. Our experiments on popular websites show that Fast Open reduces the whole-page load time over 10% on average, and in some cases up to 40%.

    Proportional Rate Reduction for TCP”, Nandita Dukkipati, Matt Mathis, Yuchung Cheng, Monia Ghobadi, Proceedings of the 11th ACM SIGCOMM Conference on Internet Measurement 2011, Berlin, Germany - November 2-4, 2011.
    Packet losses increase latency of Web transfers and negatively impact user experience. Proportional rate reduction (PRR) is designed to recover from losses quickly, smoothly and accurately by pacing out retransmissions across received ACKs during TCP’s fast recovery. Experiments on Google Web and YouTube servers in U.S. and India demonstrate that PRR reduces the TCP latency of connections experiencing losses by 3-10% depending on response size.

    Security and Privacy

    Automated Analysis of Security-Critical JavaScript APIs”, Ankur Taly, Úlfar Erlingsson, John C. Mitchell, Mark S. Miller, Jasvir Nagra, IEEE Symposium on Security & Privacy (SP), 2011.
    As software is increasingly written in high-level, type-safe languages, attackers have fewer means to subvert system fundamentals, and attacks are more likely to exploit errors and vulnerabilities in application-level logic. 
     This paper describes a generic, practical defense against such attacks, which can protect critical application resources even when those resources are partially exposed to attackers via software interfaces. In the context of carefully-crafted fragments of JavaScript, the paper applies formal methods and semantics to prove that these defenses can provide complete, non-circumventable mediation of resource access; the paper also shows how an implementation of the techniques can establish the properties of widely-used software, and find previously-unknown bugs.

    App Isolation: Get the Security of Multiple Browsers with Just One”, Eric Y. Chen, Jason Bau, Charles Reis, Adam Barth, Collin Jackson, 18th ACM Conference on Computer and Communications Security, 2011.
    We find that anecdotal advice to use a separate web browser for sites like your bank is indeed effective at defeating most cross-origin web attacks. 
     We also prove that a single web browser can provide the same key properties, for sites that fit within the compatibility constraints.

    Speech

    Improving the speed of neural networks on CPUs”, Vincent VanhouckeAndrew Senior, Mark Z. Mao, Deep Learning and Unsupervised Feature Learning Workshop, NIPS 2011.
    As deep neural networks become state-of-the-art in real-time machine learning applications such as speech recognition, computational complexity is fast becoming a limiting factor in their adoption. We show how to best leverage modern CPU architectures to significantly speed-up their inference.

    Bayesian Language Model Interpolation for Mobile Speech Input”, Cyril AllauzenMichael RileyInterspeech 2011.
    Voice recognition on the Android platform must contend with many possible target domains - e.g. search, maps, SMS. For each of these, a domain-specific language model was built by linearly interpolating several n-gram LMs from a common set of Google corpora. The current work has found a way to efficiently compute a single n-gram language model with accuracy very close to the domain-specific LMs but with considerably less complexity at recognition time.

    Statistics

    Large-Scale Parallel Statistical Forecasting Computations in R”, Murray Stokely, Farzan Rohani, Eric Tassone, JSM Proceedings, Section on Physical and Engineering Sciences, 2011.
    This paper describes the implementation of a framework for utilizing distributed computational infrastructure from within the R interactive statistical computing environment, with applications to timeseries forecasting. This system is widely used by the statistical analyst community at Google for data analysis on very large data sets.

    Structured Data

    Dremel: Interactive Analysis of Web-Scale Datasets”, Sergey Melnik, Andrey Gubarev, Jing Jing Long, Geoffrey Romer, Shiva Shivakumar, Matt Tolton, Communications of the ACM, vol. 54 (2011), pp. 114-123.
    Dremel is a scalable, interactive ad-hoc query system. By combining multi-level execution trees and columnar data layout, it is capable of running aggregation queries over trillion-row tables in seconds. Besides continued growth internally to Google, Dremel now also backs an increasing number of external customers including BigQuery and UIs such as AdExchange front-end.

    Representative Skylines using Threshold-based Preference Distributions”, Atish Das Sarma, Ashwin Lall, Danupon Nanongkai, Richard J. Lipton, Jim Xu, International Conference on Data Engineering (ICDE), 2011.
    The paper adopts principled approach towards representative skylines and formalizes the problem of displaying k tuples such that the probability that a random user clicks on one of them is maximized. This requires mathematically modeling (a) the likelihood with which a user is interested in a tuple, as well as (b) how one negotiates the lack of knowledge of an explicit set of users. This work presents theoretical and experimental results showing that the suggested algorithm significantly outperforms previously suggested approaches.

    Hyper-local, directions-based ranking of places”, Petros Venetis, Hector Gonzalez, Alon Y. Halevy, Christian S. Jensen,PVLDB, vol. 4(5) (2011), pp. 290-30.
    Click through information is one of the strongest signals we have for ranking web pages. We propose an equivalent signal for raking real world places: The number of times that people ask for precise directions to the address of the place. We show that this signal is competitive in quality with human reviews while being much cheaper to collect, we also show that the signal can be incorporated efficiently into a location search system.

    Systems

    Power Management of Online Data-Intensive Services”, David Meisner, Christopher M. Sadler, Luiz André BarrosoWolf-Dietrich Weber, Thomas F. Wenisch, Proceedings of the 38th ACM International Symposium on Computer Architecture, 2011.
    Compute and data intensive Web services (such as Search) are a notoriously hard target for energy savings techniques. This article characterizes the statistical hardware activity behavior of servers running Web search and discusses the potential opportunities of existing and proposed energy savings techniques.

    The Impact of Memory Subsystem Resource Sharing on Datacenter Applications”, Lingjia Tang, Jason Mars, Neil Vachharajani, Robert Hundt, Mary-Lou Soffa, ISCA, 2011.
    In this work, the authors expose key characteristics of an emerging class of Google-style workloads and show how to enhance system software to take advantage of these characteristics to improve efficiency in data centers. The authors find that across datacenter applications, there is both a sizable benefit and a potential degradation from improperly sharing micro-architectural resources on a single machine (such as on-chip caches and bandwidth to memory). The impact of co-locating threads from multiple applications with diverse memory behavior changes the optimal mapping of thread to cores for each application. By employing an adaptive thread-to-core mapper, the authors improved the performance of the datacenter applications by up to 22% over status quo thread-to-core mapping, achieving performance within 3% of optimal.

    Language-Independent Sandboxing of Just-In-Time Compilation and Self-Modifying Code”, Jason Ansel, Petr Marchenko, Úlfar Erlingsson, Elijah Taylor, Brad Chen, Derek Schuff, David Sehr, Cliff L. Biffle, Bennet S. Yee, ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI), 2011.
    Since its introduction in the early 90's, Software Fault Isolation, or SFI, has been a static code technique, commonly perceived as incompatible with dynamic libraries, runtime code generation, and other dynamic code. 
     This paper describes how to address this limitation and explains how the SFI techniques in Google Native Client were extended to support modern language implementations based on just-in-time code generation and runtime instrumentation. This work is already deployed in Google Chrome, benefitting millions of users, and was developed over a summer collaboration with three Ph.D. interns; it exemplifies how Research at Google is focused on rapidly bringing significant benefits to our users through groundbreaking technology and real-world products.

    Thialfi: A Client Notification Service for Internet-Scale Applications”, Atul Adya, Gregory Cooper, Daniel MyersMichael Piatek,Proc. 23rd ACM Symposium on Operating Systems Principles (SOSP), 2011, pp. 129-142.
    This paper describes a notification service that scales to hundreds of millions of users, provides sub-second latency in the common case, and guarantees delivery even in the presence of a wide variety of failures. 
     The service has been deployed in several popular Google applications including Chrome, Google Plus, and Contacts.


    翻譯進(jìn)行中.
     

    posted @ 2012-03-24 11:39 石建 | Fat Mind 閱讀(1080) | 評(píng)論 (0)編輯 收藏

    今天開始嘗試clojure,遇到的問題、經(jīng)驗(yàn)整理

    1.了解clojure
    http://metaphy.iteye.com/blog/458872

    2.開始HelloWrold
        - 搭建開發(fā)環(huán)境(對(duì)于從Java過來的人,肯定習(xí)慣eclipse)
          在線安裝的速度比烏龜還慢,推薦全手動(dòng)方式安裝插件
        (eclipse手動(dòng)安裝插件 http://www.tkk7.com/shijian/archive/2012/03/18/372141.html
          離線zip: http://roysong.iteye.com/blog/1260147
        - 跑起來
            - 先'黑窗口'吧 http://clojure.org/getting_started,熱熱身
            - eclipse開發(fā)(提醒:必須把clojure-xxx.jar加入classpath)
            - 閱讀 http://www.ibm.com/developerworks/cn/opensource/os-eclipse-clojure/,再練習(xí)

    3.如何學(xué)習(xí)
       http://weiyongqing.iteye.com/blog/1441743
        引 “我就應(yīng)該一步一步來,先把clojure的doc文檔網(wǎng)站上的core都敲打一遍,然后,學(xué)習(xí)孫寧的RPC框架,空閑時(shí)做4clojure的問題”


    posted @ 2012-03-18 23:33 石建 | Fat Mind 閱讀(1148) | 評(píng)論 (0)編輯 收藏

    一、快捷鍵

    1.常用快捷鍵
        a. crtl + h 查找內(nèi)容
        b. ctrl + shift + r 快速打開資源文件
        c. ctrl + shift + t 快速打開類文件
        d. alt + shift + o  快速打開 '選中相同詞,出現(xiàn)陰影'

    2.如何設(shè)置自己特定的快捷鍵
        


    二、插件

    請(qǐng)務(wù)必閱讀:
        http://wiki.eclipse.org/FAQ_How_do_I_install_new_plug-ins%3F (為什么推薦使用eclipse update manager)
        http://www.venukb.com/2006/08/20/install-eclipse-plugins-the-easy-way/ (主要講解'manual install'安裝 方式)

    1.插件安裝方式
        1.1 在線安裝
              官網(wǎng)wiki寫的很清楚,優(yōu)勢(shì):1.插件之間依賴管理、版本兼容性管理  2.如同你在Windows安裝軟件一樣,當(dāng)你不需要的時(shí)候可以通過update manage很容易的卸載;當(dāng)你安裝更多的plguin時(shí),更容易管理。
        eclipse wiki對(duì)manual install的看法:This obviously is a more dangerous approach, as no certification takes place about the suitability of the plug-in; it may rely on other plug-ins not available in your installation. In the case of compatibility conflicts, you won’t find out until you use the plug-in that it might break.
            可惜的是,很多時(shí)候網(wǎng)絡(luò)的情況不是很理想,嘗試很多遍后,依然失?。贿@是促使manual install根本的原因。  
        1.2 手動(dòng)安裝
            a、第一種方式:下載plugin到本地,解壓后復(fù)制features、plugin到%eclipse_home%下對(duì)應(yīng)的目錄
            如此圖 http://static.flickr.com/75/219742315_9ee663e2c8_o.png
            優(yōu)勢(shì):絕對(duì)簡單;缺點(diǎn):正好是通過update manager安裝的優(yōu)點(diǎn),插件之間的依賴、版本兼容性,以及后續(xù)的管理,都需要手動(dòng)操作。
            b、第二種方式:通過.link的方式,解決'后續(xù)管理問題'
                 b-1、eclipse目錄創(chuàng)建 links 目錄
                 b-2、創(chuàng)建對(duì)應(yīng)的.link文件,如:subversive.link
                 b-3、創(chuàng)建subversive/eclipse/,拷貝features、plugin到此目錄
                 b-4、修改subversive.link文件,如:path=E:/dev/eclipse-t/thrid-plugins/subversive
                 b-5、重啟eclipse(重啟后,發(fā)現(xiàn)要使用svn,必須安裝subversive connector;驗(yàn)證手動(dòng)安裝的缺點(diǎn))
             c、提示:
                       - 手動(dòng)安裝插件時(shí),務(wù)必仔細(xì)閱讀,此插件的先前條件(否則出問題,很難排查)。
                        如:m2eclipse先決條件
    subeclipse
    、mylyn。
                        或 “Pre-requisite: an Eclipse version including Java Support (e.g. with the JDT : Java Development Tools, as in Eclipse For Java Developers, Eclipse For RCP/RAP developers, Eclipse for JavaEE developers, etc.)” http://code.google.com/p/counterclockwise/wiki/Documentation#Install_Counterclockwise_plugin
                       - 
    eclipse 手動(dòng)安裝plugin,.link文件的path路徑 必須使用絕對(duì)路徑


    總結(jié):對(duì)eclipse插件安裝,首先推薦update manager;僅當(dāng)網(wǎng)絡(luò)環(huán)境不允許時(shí),安裝失敗時(shí),再嘗試手動(dòng)安裝。

    2.插件資源收集

    2.1、 m2eclipse插件安裝
        1)先決條件
            a、eclipse3.2或更高版本(可忽略,一般使用的eclipse已經(jīng)3.5以上 版本)
            b、jdk高于1.4版本;eclipse運(yùn)行在jdk環(huán)境,非jre環(huán)境
            c、必須先安裝插件:subeclipse(svn)、mylyn(任務(wù)管理); mylyn在eclipse3.5以上版本,已默認(rèn)存在,無需安裝
            svn插件在線安裝地址(網(wǎng)絡(luò)不確定性,更推薦下載zip,archive選擇本地文件安裝)
            http://subclipse.tigris.org/servlets/ProjectProcess;jsessionid=290480ED68C2C7E781DCCE66CE657FC2?pageID=p4wYuA
         2)安裝m2eclipse,未找到可下載到本地的zip,只能在線安裝,地址 http://www.eclipse.org/m2e/download/


    posted @ 2012-03-18 21:10 石建 | Fat Mind 閱讀(758) | 評(píng)論 (0)編輯 收藏

    題記:單元測試的過程中,遇到泛型mock的問題;重新溫習(xí)一遍,閱讀(core java 泛型)



    xmind格式(可下載) :整理過程中,記錄為xmind格式

    單元測試遇到的問題,簡化后如下:

     1     public List<? extends Date> getDateT() {
     2         return null;
     3     }
     4     public List<Date> getDate() {
     5         return null;
     6     }
     7     public void mockGetDate() {
     8         TestMain main = mock(TestMain.class);
     9         when(main.getDate()).thenReturn(new ArrayList<Date>()); //編譯OK
    10         /*
    11          * The method thenReturn(List<capture#2-of ? extends Date>) in the type 
    12          * OngoingStubbing<List<capture#2-of ? extends Date>>
                     is not applicable for the arguments (ArrayList<Date>)
    13          */
    14         when(main.getDateT()).thenReturn(new ArrayList<Date>()); //編譯錯(cuò)誤
    15         when(main.getDateT()).thenReturn(new ArrayList<Timestamp>()); //編譯錯(cuò)誤
    16         when(main.getDateT()).thenReturn(new ArrayList<Object>()); //編譯錯(cuò)誤
    17         when(main.getDateT()).thenReturn(new ArrayList()); //編譯OK
    18     }

    仍沒理解,哪位大仙,能幫我解釋下 ?
    posted @ 2012-03-08 21:05 石建 | Fat Mind 閱讀(332) | 評(píng)論 (0)編輯 收藏

    1.應(yīng)用 jar 沖突
        log4j沖突導(dǎo)致,應(yīng)用報(bào)錯(cuò)。類型轉(zhuǎn)換沖突。
        需求:定位某個(gè)類實(shí)際從那個(gè)jar加載 ? -verbose:class 參數(shù)(或者 
    -XX:+TraceClassLoading),詳細(xì)的記錄了加載了那些類、從那個(gè)jar加載。

    參見:http://agapple.iteye.com/blog/946603

    2.性能測試過程
       linux有什么命令、或軟件,可以同時(shí)收集cpu、load、上下文切換、mem、網(wǎng)絡(luò)IO、磁盤IO等數(shù)據(jù)嗎 ?
       vmstat 含義詳解 ? ->  圖形化報(bào)表 (痛苦的是要'人工'看著記錄數(shù)據(jù),這簡直是程序員的污點(diǎn)呀)
       (vmstat的IO統(tǒng)計(jì)的是塊設(shè)備(如磁盤)的數(shù)據(jù),網(wǎng)卡沒有對(duì)應(yīng)的設(shè)備文件(http://oss.org.cn/kernel-book/ch11/11.2.3.htm),網(wǎng)絡(luò)IO統(tǒng)計(jì)使用iftop) 
       vmstat http://linux.about.com/library/cmd/blcmdl8_vmstat.htm

    3.Jboss啟動(dòng)錯(cuò)誤 
    java.sql.SQLException: Table already exists: JMS_MESSAGES in statement [CREATE CACHED TABLE JMS_MESSAGES]
    參見:http://dinghaoliang.blog.163.com/blog/static/126540714201082764733272/
    %jboss_home%/server/default/deploy/hsqldb-ds.xml這個(gè)文件中有一個(gè)DefaultDS數(shù)據(jù)源配置,臨時(shí)解決刪除hsqldb-ds.xml文件。原因未知。

    4.logback 0.9.19 版本,引入<encoder>,放棄 <appender><layout></appenader>
            <encoder>
                
    <pattern>%m%n</pattern>
                
    <charset class="java.nio.charset.Charset">UTF-8</charset>
            
    </encoder>

    源碼:OutputStreamAppender.java
      protected void writeOut(E event) throws IOException {
        this.encoder.doEncode(event);
      }
    對(duì)日志文件charset指定,經(jīng)過debug調(diào)試,必須通過此方式配置才有效。否則取系統(tǒng)默認(rèn)編碼。

    5.設(shè)置linux系統(tǒng)編碼
    http://linux.vbird.org/linux_basic/0320bash.php#variable_locale
    其實(shí)‘系統(tǒng)編碼’設(shè)置,即設(shè)置對(duì)應(yīng)的系統(tǒng)變量,則所有可設(shè)置系統(tǒng)變量的文件都可設(shè)置編碼,export使其生效
    locale 查看當(dāng)前用戶使用的編碼(),locale -a 查看機(jī)器所支持的所有編碼
    默認(rèn)設(shè)置:
      a、系統(tǒng)級(jí)別  /etc/profile -> /etc/sysconfig/i18n,設(shè)置 LANG (無效顯示export生效)(YY:i18n有個(gè)LANGUAGE設(shè)定,不知其含義,刪除無影響)
      b、用戶級(jí)別 ~/bash_rc、~/bash_profile、~/bash_login、~/profile,讀取有限順序:從左向右;必須顯示export生效
      
    設(shè)定 LANG 或者是 LC_ALL 時(shí),則其他的語系變數(shù)就會(huì)被這兩個(gè)變數(shù)所取代。總之一句話:在當(dāng)前用戶設(shè)置LANG,是最優(yōu)方案。

    posted @ 2011-12-15 15:55 石建 | Fat Mind 閱讀(715) | 評(píng)論 (0)編輯 收藏

    現(xiàn)象與思路 :

    1.業(yè)務(wù)變化快,導(dǎo)致需要不斷溝通 ?
        a、開始1、2功能一起,后來拆分開,先上1;再到1功能,各團(tuán)隊(duì)不一致上線  (答:難解,要更有產(chǎn)品意識(shí),幫助產(chǎn)品進(jìn)行分析)
    2.開發(fā):
        a、原代碼不優(yōu)雅,總有想重構(gòu)的沖動(dòng) (答:想是白想,一定要有結(jié)果)
        b、相互等待,接口定義不明確,如:接口jar無法提供,注釋不明確  (答:時(shí)間點(diǎn)&白紙黑字,先溝通再找主管)
        c、聯(lián)調(diào)準(zhǔn)備不充分,如:hsf 因?yàn)镮P發(fā)布失敗,主站頁面配置被修改 (答:前提明確要求,不能有邏輯問題)
    3.上線
        a、涉及面太廣,信息溝通丟失,如:只是暫停審核,演繹成 '停止所有crm操作'  (答:一定要直接周知到所有相關(guān)人)
        b、風(fēng)險(xiǎn)點(diǎn)評(píng)估的不全面,未全盤考慮,如:simbacall->bp參數(shù)對(duì)象不一致,是否會(huì)導(dǎo)致失敗未聯(lián)調(diào) (答:不要遺漏或輕視任何風(fēng)險(xiǎn)點(diǎn))


    2011.11.18 晚 19:30
    posted @ 2011-11-18 19:39 石建 | Fat Mind 閱讀(173) | 評(píng)論 (0)編輯 收藏

    Annotation

     

    題記:建議關(guān)于spring問題,請(qǐng)記得查看spring reference。

     

    一、annotation前生后世

    Annotations do not directly affect program semantics, but they do affect the way programs are treated by tools and libraries, which can in turn affect the semantics of the running program. Annotations can be read from source files, class files, or reflectively at run time.

    譯:annotation不會(huì)直接影響程序的語義,xxx。Annotation可以從源文件、class文件、通過反射在運(yùn)行時(shí)讀取。

     

    參考:http://www.developer.com/print.php/3556176

    1.       定義

    Annotation type declarations are similar to normal interface declarations. An at-sign (@) precedes the interface keyword. Each method declaration defines an element of the annotation type. Method declarations must not have any parameters or a throws clause. Return types are restricted to primitives, StringClassenums, annotations, and arrays of the preceding types. Methods can have default values.

    Annotation聲明與普通的interface非常相似,在關(guān)鍵字interface前加@。每一個(gè)方法的聲明定義一個(gè)annotation的元素。方法不能有任何的參數(shù)或throws異常。返回類型被限制為:原始類型、String、Class、enum、annotation、前面描述的類型組成的數(shù)組。method定義允許有默認(rèn)值。

     

    2.       Java annotation

    There are two types of annotations available with JDK5:

    1) Simple annotations: These are the basic types supplied with Tiger, which you can use to annotate your code only; you cannot use those to create a custom annotation type.

    三個(gè)基本的annotation,如:OverrideDeprecated、Suppresswarnings,不能使用它去定義新的annotation。

    2) Meta annotations: These are the annotation types designed for annotating annotation-type declarations. Simply speaking, these are called the annotations-of-annotations.

    annotation,定義annotation的類型。如:TargetRetention、Documented、Inherited

    A.      Target:聲明annotation注解的目標(biāo)類型。如@Target(ElementType.TYPE)、@Target(ElementType.METHOD)

    B.      Retention:聲明annotation被保留的長度。如:RetentionPolicy.SOURCE、RetentionPolicy.CLASS、RetentionPolicy.RUNTIME

    C.      Documented:聲明被注解的target生成doc是否需要顯示annotation信息。

    D.      Inheritedxxx

     

    3.       annotation作用

    a. 不影響程序原本語義的情況下,增加信息+工具=聲明式編程。如:spring aop

    b. 編譯檢查

     

     

    二、利用annotation實(shí)現(xiàn)aop

    1. 思路

    A.自定義注解定義規(guī)則(何時(shí)執(zhí)行)

    B.標(biāo)記行為(執(zhí)行什么)

    C.通過反射生成代理,在對(duì)象執(zhí)行任何方法時(shí),進(jìn)行攔截判斷是否符合規(guī)則;若符合,執(zhí)行對(duì)應(yīng)的行為。

    2. 例子

    場景:一個(gè)表演家,表演節(jié)目后,觀眾拍手鼓掌

    原始:表演家擁有所有觀眾的引用,在自己表演完后,通知觀眾鼓掌

    問題:表演家應(yīng)該關(guān)注表演自身的事情,有哪些觀眾、觀眾是否鼓掌,不是其所關(guān)注的

    改進(jìn):AOP方式,表演家僅關(guān)注表演,觀眾鼓掌由其它人負(fù)責(zé)

    總結(jié):

    面向切面編程 (AOP) 提供從另一個(gè)角度來考慮程序結(jié)構(gòu)以完善面向?qū)ο缶幊蹋?/span>OOP)。 面向?qū)ο髮?yīng)用程序分解成各個(gè)層次的對(duì)象,而AOP將程序分解成各個(gè)切面或者說關(guān)注點(diǎn)。這使得可以模塊化諸如事務(wù)管理等這些橫切多個(gè)對(duì)象的關(guān)注點(diǎn)。

     

    三、spring aop如何使用annotation

    1. 基本方式,通過ProxyFactoryBean

    a.通知

    b.切入點(diǎn)

    b.通知+切入點(diǎn) à 切面

    c.實(shí)際對(duì)象+接口+切面 à 接口代理()

    2. 自動(dòng)代理方式:

    基本方式的問題是,xml配置文件繁瑣。

    1)基于spring上下文的通知者Bean的自動(dòng)代理(利用PostBeanProcessor ?)

    BeanNameAutoProxyCreator,有屬性:beanNames(被代理對(duì)象)、interceptorNames(通知)。自動(dòng)代理beanNames指定的bean,此時(shí)interceptorNames指定通知,但interceptor需要實(shí)現(xiàn)特定的接口,如:MethodBeforeAdvice。

    2) 基于aspectJ注解驅(qū)動(dòng)

    使用aspectJ風(fēng)格的注解。原理:生成代理,僅方法級(jí)別。使用AspectJ 提供的一個(gè)庫來做切點(diǎn)(pointcut)解析和匹配。

    4.       <aop:config>,優(yōu)點(diǎn):任何類都能轉(zhuǎn)化為切面,不需要特殊接口或注解,原始pojo對(duì)象

    5.       aop參數(shù)傳遞,見:spring reference 6.3.3.6 通知參數(shù)

    a. around切點(diǎn),必須傳遞ProceedingJoinPoint參數(shù),通過ProceedingJoinPoint取得方法的所有信息。

    b. 其它切點(diǎn),利用JoinPoint取得方法的所有參數(shù)。

    c. 通過參數(shù)名綁定傳遞參數(shù),未理解。

     

     

    四、annotation相關(guān)技術(shù)

    1. cglib

    它的原理就是用Enhancer生成一個(gè)原有類的子類,并且設(shè)置好callbackproxy, 則原有類的每個(gè)方法調(diào)用都會(huì)轉(zhuǎn)為調(diào)用實(shí)現(xiàn)了MethodInterceptor接口的proxyintercept() 函數(shù)。

    2.  asm

    ASM is an all purpose Java bytecode manipulation and analysis framework. It can be used to modify existing classes or dynamically generate classes, directly in binary form. Provided common transformations and analysis algorithms allow to easily assemble custom complex transformations and code analysis tools.

    ASM offer similar functionality as other bytecode frameworks, but it is focused on simplicity of use and performance. 

    簡單理解asm是比cglib更高級(jí)的code generate lib。

     

    五、others問題

    1. Generics 

    Generics - This long-awaited enhancement to the type system allows a type or method to operate on objects of various types while providing compile-time type safety. It adds compile-time type safety to the Collections Framework and eliminates the drudgery of casting. See theGenerics Tutorial. (JSR 14)

    泛型提供了編譯時(shí)類型檢查安全。這點(diǎn)很重要嗎 ?

     

    2. 為什么spring aop aspectJ anonotationpointcut聲明,必須注解在方法上 ?通過方法唯一標(biāo)識(shí)一個(gè)pointcut

    猜測:@PointcutTarget屬性指定僅為method。方法簽名成為pointcutid

      

     

    posted @ 2011-06-25 20:09 石建 | Fat Mind 閱讀(332) | 評(píng)論 (0)編輯 收藏
    參照:http://www.quora.com/What-are-some-interesting-and-innovative-startups-in-China

    2011年的中國互聯(lián)網(wǎng)還有什新奇的事發(fā)生嗎 ?

    1.團(tuán)購:美團(tuán)、拉手等(燒錢階段,搶用戶、流量,用戶體驗(yàn)必須提升,如:退款等服務(wù);預(yù)測:再有1年時(shí)間,團(tuán)購巨頭就會(huì)形成)
    2.在線租書 : booksfly(有創(chuàng)意,但目前的運(yùn)營方式很難有大的成長)
    3.圖片搜索 : taotaosou (有創(chuàng)意和前途,目前:用戶體驗(yàn)差、效果差)

    4.微?。盒吕?、騰訊(蒸蒸日上,新浪占據(jù)主導(dǎo)地位)
    5.垂直化社交網(wǎng)站:Mtime、果殼(新方向,垂直化的社交會(huì)去慢慢蠶食)
    6.rss服務(wù):鮮果(無自己特點(diǎn))

    7.網(wǎng)頁游戲 : 忍者村、三國殺(賺錢的行業(yè),但游戲生命周期短的特點(diǎn)值得考慮)
    8.在線音樂 : 豆瓣電臺(tái)、酷狗(推崇豆瓣的簡約,做到了極致)

    9.在線存儲(chǔ) : 微盤、DBBank

    10.手機(jī)應(yīng)用:豌豆莢、VivaMe(閱讀器\傳媒)、街旁(lbs)
    11.UC樂園(UC社交平臺(tái),盡管UCWeb是入口,有著先天優(yōu)勢(shì))



     PS:以下純屬個(gè)人YY,隨手記錄

    1.親子、育兒教育社區(qū) + 電商,如何做 ?淘寶http://t.cn/blAkG 全是tms靜態(tài)頁面[囧] 專業(yè)親子網(wǎng)站排名 http://t.cn/SwZSXr


    posted @ 2011-04-16 13:08 石建 | Fat Mind 閱讀(197) | 評(píng)論 (0)編輯 收藏

    主要參考:構(gòu)建高性能web站點(diǎn)


    一、網(wǎng)卡

    網(wǎng)卡使用一個(gè)特定的物理層數(shù)據(jù)鏈路層標(biāo)準(zhǔn),例如以太網(wǎng)來實(shí)現(xiàn)通訊所需要的電路系統(tǒng)。這為一個(gè)完整的網(wǎng)絡(luò)協(xié)議棧提供了基礎(chǔ),使得在同一局域網(wǎng)中的小型計(jì)算機(jī)組以及通過路由協(xié)議連接的廣域網(wǎng),例如IP,都能夠進(jìn)行通訊。

    1.       作用:

    a)         唯一的mac地址,定位機(jī)器(局域網(wǎng)/以太網(wǎng)mac尋址)

    b)        數(shù)據(jù)接收和發(fā)送。擁有物理緩存區(qū)。

                             i.              接收:接收物理層數(shù)據(jù),通過DMA方式訪問內(nèi)存。

                           ii.              發(fā)送:接收上層數(shù)據(jù),分解為適當(dāng)大小的數(shù)據(jù)包發(fā)送。

    轉(zhuǎn)載:

    數(shù)據(jù)的封裝與解封:發(fā)送時(shí)將上一層交下來的數(shù)據(jù)加上首部和尾部,成為以太網(wǎng)的幀。接收時(shí)將以太網(wǎng)的幀剝?nèi)ナ撞亢臀膊?,然后送交上一層?/span>
    鏈路管理:主要是CSMA/CDCarrier Sense Multiple Access with Collision Detection ,帶沖突檢測的載波監(jiān)聽多路訪問)協(xié)議的實(shí)現(xiàn)。
    編碼與譯碼:即曼徹斯特編碼與譯碼。

    2. 協(xié)議

    以太網(wǎng)Ethernet)是一種計(jì)算機(jī)局域網(wǎng)組網(wǎng)技術(shù)。

    ARP協(xié)議Address Resolution Protocol),或稱地址解析協(xié)議。ARP協(xié)議的基本功能就是通過目標(biāo)設(shè)備的IP地址,查詢目標(biāo)設(shè)備的MAC地址。

    http://zh.wikipedia.org/zh/%E5%9C%B0%E5%9D%80%E8%A7%A3%E6%9E%90%E5%8D%8F%E8%AE%AE

    3. 傳輸速率

    網(wǎng)卡速率是指網(wǎng)卡每秒鐘接收或發(fā)送數(shù)據(jù)的能力,單位是Mbps(兆位/秒)由于存在多種規(guī)范的以太網(wǎng),所以網(wǎng)卡也存在多種傳輸速率,以適應(yīng)它所兼容的以太網(wǎng)。目前網(wǎng)卡在標(biāo)準(zhǔn)以太網(wǎng)中速度為10Mbps,在快速以太網(wǎng)中速度為100Mbps,在千兆以太網(wǎng)中速度為1000Mbps等。

    主流的網(wǎng)卡主要有10Mbps網(wǎng)卡、100Mbps以太網(wǎng)卡、10Mbps/100Mbps自適應(yīng)網(wǎng)卡、1000Mbps千兆以太網(wǎng)卡以及最新出現(xiàn)的萬兆網(wǎng)卡五種。對(duì)于一般家庭用戶選購10M或者10Mbps/100Mbps自適應(yīng)網(wǎng)卡即可,對(duì)于企業(yè)用戶建議購買100Mbps以太網(wǎng)卡或者1000Mbps千兆以太網(wǎng)卡或者萬兆網(wǎng)卡。

    以太網(wǎng)卡和交換設(shè)備都支持多速率,設(shè)備之間通過自動(dòng)協(xié)商設(shè)置最佳的連接速度和雙工方式。如果協(xié)商失敗,多速率設(shè)備就會(huì)探測另一方使用的速率但是默認(rèn)為半雙工方式。10/100以太網(wǎng)端口支持10BASE-T100BASE-TX。

    2.       特點(diǎn)

    a)         全雙工

    b)        傳輸速率

    c)        總線類型:PCI總線架構(gòu)日益成為網(wǎng)卡的首選總線

    d)        MAC地址

    二、數(shù)據(jù)如何發(fā)送

    1.       將數(shù)據(jù)寫入用戶進(jìn)程的內(nèi)存地址空間,其實(shí)實(shí)際的開發(fā)過程只需對(duì)運(yùn)行時(shí)變量賦值即可

    2.       應(yīng)用程度調(diào)用系統(tǒng)函數(shù),將數(shù)據(jù)從用戶態(tài)內(nèi)存區(qū)復(fù)制到由內(nèi)核維護(hù)的一段稱為內(nèi)核緩沖區(qū)的內(nèi)存地址空間。

    a)         內(nèi)核緩存區(qū)大小有限,要發(fā)送的數(shù)據(jù)以隊(duì)列的形式進(jìn)入

    b)        每次復(fù)制一定的數(shù)據(jù)大小,這個(gè)大小取決于網(wǎng)絡(luò)數(shù)據(jù)包的大小以及內(nèi)核緩存區(qū)的承載能力

    3.       當(dāng)數(shù)據(jù)寫入內(nèi)核緩存區(qū),內(nèi)核會(huì)通知網(wǎng)卡控制器來讀取數(shù)據(jù),cpu轉(zhuǎn)而處理其它任務(wù)

    a)         網(wǎng)卡將發(fā)送的數(shù)據(jù)從內(nèi)核緩存區(qū)復(fù)制到網(wǎng)卡緩存區(qū)

    b)        數(shù)據(jù)的復(fù)制始終按照內(nèi)部總線的寬度復(fù)制(如32位總線,每次復(fù)制32bit信息)

    4.       網(wǎng)卡發(fā)送數(shù)據(jù)到物理線路

    a)         需要對(duì)數(shù)據(jù)進(jìn)行字節(jié)到位的轉(zhuǎn)換(即將數(shù)據(jù)按照位的順序發(fā)出)

    b)        網(wǎng)卡內(nèi)部使用特定的物理裝置,來生成可以傳播的各種信息,如銅線,網(wǎng)卡會(huì)根據(jù)位信息“0/1的變化產(chǎn)生不同的電信號(hào);光線,網(wǎng)卡會(huì)生成光信號(hào)。

    三、電磁波速度

           不管是電信號(hào),還是光信號(hào),進(jìn)入物理介質(zhì)后,其傳輸速度僅依賴其傳播介質(zhì),銅線中電信號(hào)的傳輸速度大約2.3*108m/s,光纖中光信號(hào)的傳播速度大約是2.0*108m/s。光在真空中的傳播速度是3.0*108m/s,為什么光纖中的傳播速度要慢呢 ?因?yàn)楣庠诠饫w中的傳播利用全反射原理,所以傳播距離要大于光纖長度。

           由此看見,不同的傳播介質(zhì)中信號(hào)的傳播速度幾乎是常量。也就是說,不論數(shù)據(jù)發(fā)送裝置以多快的發(fā)送速度讓數(shù)據(jù)以信號(hào)的形式進(jìn)入路線,在線路中信號(hào)的傳播速度幾乎可以認(rèn)為是一樣快的。

           光纖與銅線相比?光纖采用全反射原理,因此光信號(hào)衰減底,因此傳播距離遠(yuǎn)。

    四、帶寬概念

           從上面分析來看,數(shù)據(jù)的傳輸包括:發(fā)送端發(fā)送數(shù)據(jù)進(jìn)入線路 + 線路傳輸,線路傳輸?shù)乃俣仍诟鞣N傳輸介質(zhì)幾乎是相同的。

           帶寬定義:每秒傳播bit數(shù),bit/s。

           這樣看,影響帶寬的因素僅為“發(fā)送端發(fā)送數(shù)據(jù)進(jìn)入線路”,如何提升:a、提升發(fā)送速度 b、數(shù)據(jù)傳輸?shù)牟⑿卸?/span>

    1.       發(fā)送速度

    數(shù)據(jù)發(fā)送裝置將二進(jìn)制信號(hào)傳送至線路的能力。關(guān)鍵是,如果接收能力跟不上,發(fā)送能力不可能提高。原理:接收速度決定發(fā)送速度。

           也就是“流控機(jī)制”,保證接收方能夠接收數(shù)據(jù),不會(huì)丟失數(shù)據(jù)。如Tcp滑動(dòng)窗口(滑動(dòng)窗口協(xié)議的基本原理,任意時(shí)刻發(fā)送方、接收方都保持一個(gè)連續(xù)的允許發(fā)送、接收的幀的序號(hào)http://blog.csdn.net/yujun00/archive/2006/03/23/636495.aspx)。

    2.    并行度,等價(jià)于計(jì)算機(jī)總線的概念。比如:32位,任意同一時(shí)刻能傳輸32位數(shù)據(jù)。

    總結(jié):顯然,網(wǎng)卡影響性能結(jié)果。

     

    posted @ 2011-04-04 00:17 石建 | Fat Mind 閱讀(342) | 評(píng)論 (0)編輯 收藏

    題記:在淘寶廣告技術(shù)部工作快1年,寫點(diǎn)自己對(duì)廣告的認(rèn)識(shí)

    目前在淘寶主要存在這樣幾種形式廣告 CPT、CPC、CPS

    1.CPT
    cost per time
    按時(shí)長計(jì)費(fèi)。大部分屬于品牌廣告,主要著重于品牌形象的宣傳。比如:淘寶中屏滾動(dòng)廣告,如“dell、九牧王。特點(diǎn):a、價(jià)格非常貴 b、位置少。
    CPM
    Cost Per ThousandImpression)按千次展現(xiàn)次數(shù)計(jì)費(fèi)。能夠?yàn)閺V告主帶來穩(wěn)定的廣告展現(xiàn),但效果是未知數(shù),具體要看投放的媒體,以及場景的相關(guān)性。能為有流量的站點(diǎn)帶來穩(wěn)定的收入。

    2.CPC
    cost per click
    按點(diǎn)擊計(jì)費(fèi)。如:google ads、淘寶直通車,都是根據(jù)詞的競價(jià)排名,決定展示那個(gè)廣告主的廣告。
    目前淘寶大部分收入來自于直通車。
    如:在淘寶,搜諾基亞 N73”,首先做搜索詞歸一化,匹配為與競價(jià)詞相關(guān)的詞,再根據(jù)競價(jià)詞去搜索。誰出價(jià)高,并根據(jù)用戶的信譽(yù)度等因素,決定出誰的廣告。
    站在長遠(yuǎn)的發(fā)展,淘寶直通車的目的:1.增加淘寶收入 2.促進(jìn)成交。如果僅僅是點(diǎn)擊最大化,增加淘寶的收入,但并沒有為賣家?guī)沓山?,則賣家的出價(jià)必然會(huì)降低,對(duì)于雙方來說都是雙輸的局面。所以必須站在促進(jìn)成交的前提下,通過直通車不斷提升淘寶自身的收入增加。

    3.CPS
    cost per Sales
    按成交計(jì)費(fèi)。據(jù)說:某淘寶廣告去日本時(shí)在一家書店看到的營銷模式,回到淘寶后,決定做"淘寶客"。

    cps模式更加關(guān)注的是“長尾流量”,因?yàn)殚L尾publisher,流量質(zhì)量一般來說不是很高,如果按照前面說到的3種方式計(jì)費(fèi),對(duì)廣告主來說是不公平的。但根據(jù)成交,分成給長尾publisher,既能保證廣告主的利益,也滿足廣告主營銷、推廣的目的。同時(shí)也能夠給publisher帶來收入。


    小思考:

        目前淘寶有
    8億商品,僅淘寶自身的廣告位置,完全不能滿足賣家推廣、營銷的目的。所以淘寶會(huì)去買外部廣告位(如優(yōu)酷),或者與外部網(wǎng)站以“分成”的模式合作(google)。

        其實(shí),淘寶也在建立自己的廣告聯(lián)盟,類似于“圈地運(yùn)動(dòng)”,去累積足夠多的流量、渠道,到時(shí)怎么玩,都是由自己決定。

    其實(shí)目前的淘寶廣告外投,對(duì)于客戶來說是不透明的,賣家不知道自己的廣告是否被外投,與淘寶站內(nèi)的廣告相比,站外的廣告效果還是要差很多(會(huì)基于成交效果給予廣告主打折),準(zhǔn)確的說“淘寶的廣告主是被外投”。

        其實(shí),目前淘寶聯(lián)盟,對(duì)于廣告主與網(wǎng)站主自主選擇僅有CPT計(jì)費(fèi)模式,當(dāng)廣告位無廣告時(shí)淘寶聯(lián)盟會(huì)自動(dòng)推送CPS模式廣告。CPC模式廣告,在淘寶聯(lián)盟目前的網(wǎng)站主管理模式下,無法推廣CPC廣告。原因是,目前網(wǎng)站主沒有明確的層級(jí)結(jié)構(gòu),對(duì)于CPC廣告,流量好壞決定最后的成交率,相比CPT模式CPC更加賺錢,大量流量差的投放CPC廣告,會(huì)導(dǎo)致cpc(單次點(diǎn)擊消耗)降低,肯定對(duì)淘寶廣告的收入會(huì)有非常大的影響。淘寶如果要開放CPC模式給外部網(wǎng)站主,必須有明確的層級(jí)結(jié)構(gòu)、站內(nèi)&站外競價(jià)區(qū)分。

        外投每天能為淘寶帶來大約300萬收入,但對(duì)于主動(dòng)獲取廣告的publisher來說沒有智能匹配模式、沒有靈活的定制化界面、多樣的創(chuàng)意。當(dāng)站內(nèi)CPC趨于飽和時(shí),如何開拓好外投,對(duì)于淘寶直通車則是必然的趨勢(shì)。而CPS自然去占據(jù)長尾流量,CPC如何占據(jù)主外部中型網(wǎng)站,則是至關(guān)重要的。


    posted @ 2011-03-20 20:01 石建 | Fat Mind 閱讀(501) | 評(píng)論 (0)編輯 收藏
         摘要:  一、Java數(shù)據(jù)類型   數(shù)據(jù)類型就是對(duì)內(nèi)存位置的抽象表達(dá) (很多編程語言都依賴于特定的計(jì)算機(jī)類型和對(duì)數(shù)據(jù)類型屬性的具體編譯實(shí)現(xiàn),比如word和integer數(shù)據(jù)類型的大小等;Java通過JVM保證數(shù)據(jù)所占存儲(chǔ)空間的大小不會(huì)隨硬件的改變發(fā)生變化)。 1.   Primitive data type :A primiti...  閱讀全文
    posted @ 2010-12-18 16:52 石建 | Fat Mind 閱讀(284) | 評(píng)論 (0)編輯 收藏

    導(dǎo)航

    <2010年12月>
    2829301234
    567891011
    12131415161718
    19202122232425
    2627282930311
    2345678

    統(tǒng)計(jì)

    常用鏈接

    留言簿

    隨筆分類

    隨筆檔案

    搜索

    最新評(píng)論

    What 、How、Why,從細(xì)節(jié)中尋找不斷的成長點(diǎn)
    主站蜘蛛池模板: 免费日本一区二区| 色www永久免费视频| 成人福利免费视频| 国产国产人免费人成免费视频 | 亚洲黄色三级网站| 亚洲女子高潮不断爆白浆| 一个人晚上在线观看的免费视频| 蜜桃成人无码区免费视频网站 | 国产AV无码专区亚洲AV蜜芽 | 最新亚洲春色Av无码专区| 91视频免费观看| 免费观看一级毛片| 亚洲另类无码专区首页| 在线精品免费视频| 久久久久亚洲AV无码观看| 99久久久国产精品免费牛牛四川 | 免费无遮挡无遮羞在线看| 亚洲高清中文字幕免费| 亚洲va久久久噜噜噜久久男同| 亚洲高清一区二区三区电影| 最近免费视频中文字幕大全| 亚洲自偷自偷在线制服 | 啦啦啦www免费视频| 亚洲av中文无码字幕色不卡| 免费一级毛片正在播放| 亚洲色欲啪啪久久WWW综合网| 亚欧在线精品免费观看一区| 久久精品国产亚洲AV麻豆不卡| 亚洲免费视频网站| 亚洲AV无码日韩AV无码导航| 99精品一区二区免费视频| 亚洲手机中文字幕| a级毛片在线免费| 亚洲永久永久永久永久永久精品| 国产一二三四区乱码免费| 国产91久久久久久久免费| 亚洲AV香蕉一区区二区三区| 毛片a级毛片免费观看免下载| 中文字幕在线观看亚洲| 啦啦啦中文在线观看电视剧免费版 | 成年人免费网站在线观看|