你好,游客 登录 注册 搜索
背景:
阅读新闻

【hadoop2.7.1】Writable的方方面面

[日期:2015-10-16] 来源:CSDN博客  作者: [字体: ]

由于Hadoop的MapReduce和HDFS都有通信的需求,需要对通信的对象进行序列化。Hadoop并没有采用Java的序列化,而是基于java.io里的DataOutput和DataInput引入了它自己的系统,一个简单高效的序列化协议。

org.apache.hadoop.io中定义了大量的可序列化对象,他们都实现了Writable接口。

我们先来看hadoop2.7.1中Writable接口的源码:

package org.apache.hadoop.io;

import java.io.DataOutput;
import java.io.DataInput;
import java.io.IOException;

import org.apache.hadoop.classification.InterfaceAudience;
import org.apache.hadoop.classification.InterfaceStability;

/**
 * A serializable object which implements a simple, efficient, serialization 
 * protocol, based on {@link DataInput} and {@link DataOutput}.
 *
 * <p>Any <code>key</code> or <code>value</code> type in the Hadoop Map-Reduce
 * framework implements this interface.</p>
 * 
 * <p>Implementations typically implement a static <code>read(DataInput)</code>
 * method which constructs a new instance, calls {@link #readFields(DataInput)} 
 * and returns the instance.</p>
 * 
 * <p>Example:</p>
 * <p><blockquote><pre>
 *     public class MyWritable implements Writable {
 *       // Some data     
 *       private int counter;
 *       private long timestamp;
 *       
 *       public void write(DataOutput out) throws IOException {
 *         out.writeInt(counter);
 *         out.writeLong(timestamp);
 *       }
 *       
 *       public void readFields(DataInput in) throws IOException {
 *         counter = in.readInt();
 *         timestamp = in.readLong();
 *       }
 *       
 *       public static MyWritable read(DataInput in) throws IOException {
 *         MyWritable w = new MyWritable();
 *         w.readFields(in);
 *         return w;
 *       }
 *     }
 * </pre></blockquote></p>
 */
@InterfaceAudience.Public
@InterfaceStability.Stable
public interface Writable {
  /** 
   * Serialize the fields of this object to <code>out</code>.
   * 
   * @param out <code>DataOuput</code> to serialize this object into.
   * @throws IOException
   */
  void write(DataOutput out) throws IOException;

  /** 
   * Deserialize the fields of this object from <code>in</code>.  
   * 
   * <p>For efficiency, implementations should attempt to re-use storage in the 
   * existing object where possible.</p>
   * 
   * @param in <code>DataInput</code> to deseriablize this object from.
   * @throws IOException
   */
  void readFields(DataInput in) throws IOException;
}

在源码的开始,是对Writable接口的一个介绍,并且实现了Writable接口的一个典型例子。

接口类包含有两个注解,而注解的作用主要是用来标示一些类和方法,下面分别做介绍:

InterfaceAudience类

主要用于说明使用的范围:

interfaceAudience 类包含三个注解类型,用来被说明被他们注解的类型的潜在的使用范围(audience)。

@InterfaceAudience.Public: 对所有工程和应用可用

@InterfaceAudience.LimitedPrivate: 仅限于某些特定工程,如Comomn,HDFS等

@InterfaceAudience.Private: 仅限于Hadoop

在hadoop2.7.1中其实现类如下:

package org.apache.hadoop.classification;

import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;

/**
 * Annotation to inform users of a package, class or method's intended audience.
 * Currently the audience can be {@link Public}, {@link LimitedPrivate} or
 * {@link Private}. <br>
 * All public classes must have InterfaceAudience annotation. <br>
 * <ul>
 * <li>Public classes that are not marked with this annotation must be
 * considered by default as {@link Private}.</li> 
 * 
 * <li>External applications must only use classes that are marked
 * {@link Public}. Avoid using non public classes as these classes
 * could be removed or change in incompatible ways.</li>
 * 
 * <li>Hadoop projects must only use classes that are marked
 * {@link LimitedPrivate} or {@link Public}</li>
 * 
 * <li> Methods may have a different annotation that it is more restrictive
 * compared to the audience classification of the class. Example: A class 
 * might be {@link Public}, but a method may be {@link LimitedPrivate}
 * </li></ul>
 */
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class InterfaceAudience {
  /**
   * Intended for use by any project or application.
   */
  @Documented
  @Retention(RetentionPolicy.RUNTIME)
  public @interface Public {};
  
  /**
   * Intended only for the project(s) specified in the annotation.
   * For example, "Common", "HDFS", "MapReduce", "ZooKeeper", "HBase".
   */
  @Documented
  @Retention(RetentionPolicy.RUNTIME)
  public @interface LimitedPrivate {
    String[] value();
  };
  
  /**
   * Intended for use only within Hadoop itself.
   */
  @Documented
  @Retention(RetentionPolicy.RUNTIME)
  public @interface Private {};

  private InterfaceAudience() {} // Audience can't exist on its own
}

InterfaceStability 类

包含三个注解,用于说明被他们注解的类型的稳定性,及有特定包的依赖:

@InterfaceStability.Stable: 主版本是稳定的,不同主版本间可能不兼容

@InterfaceStability.Evolving: 不断变化,不同次版本间可能不兼容

@InterfaceStability.Unstable: 没有任何可靠性和健壮性保证,即不能提供稳定性和可靠性

在hadoop2.7.1中其实现类如下:

package org.apache.hadoop.classification;

import java.lang.annotation.Documented;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;

import org.apache.hadoop.classification.InterfaceAudience.LimitedPrivate;
import org.apache.hadoop.classification.InterfaceAudience.Private;
import org.apache.hadoop.classification.InterfaceAudience.Public;

/**
 * Annotation to inform users of how much to rely on a particular package,
 * class or method not changing over time. Currently the stability can be
 * {@link Stable}, {@link Evolving} or {@link Unstable}. <br>
 * 
 * <ul><li>All classes that are annotated with {@link Public} or
 * {@link LimitedPrivate} must have InterfaceStability annotation. </li>
 * <li>Classes that are {@link Private} are to be considered unstable unless
 * a different InterfaceStability annotation states otherwise.</li>
 * <li>Incompatible changes must not be made to classes marked as stable.</li>
 * </ul>
 */
@InterfaceAudience.Public
@InterfaceStability.Evolving
public class InterfaceStability {
  /**
   * Can evolve while retaining compatibility for minor release boundaries.; 
   * can break compatibility only at major release (ie. at m.0).
   */
  @Documented
  @Retention(RetentionPolicy.RUNTIME)
  public @interface Stable {};
  
  /**
   * Evolving, but can break compatibility at minor release (i.e. m.x)
   */
  @Documented
  @Retention(RetentionPolicy.RUNTIME)
  public @interface Evolving {};
  
  /**
   * No guarantee is provided as to reliability or stability across any
   * level of release granularity.
   */
  @Documented
  @Retention(RetentionPolicy.RUNTIME)
  public @interface Unstable {};
}

Writable接口定义了两个方法:

write实现把对象序列化的功能;

readFields实现把对象反序列化的功能。





收藏 推荐 打印 | 录入:elainebo | 阅读:
本文评论   查看全部评论 (0)
表情: 表情 姓名: 字数
点评:
       
评论声明
  • 尊重网上道德,遵守中华人民共和国的各项有关法律法规
  • 承担一切因您的行为而直接或间接导致的民事或刑事法律责任
  • 本站管理人员有权保留或删除其管辖留言中的任意内容
  • 本站有权在网站内转载或引用您的评论
  • 参与本评论即表明您已经阅读并接受上述条款