publicinterfaceReadWriteLock { /** * Returns the lock used for reading. * * @return the lock used for reading. */ LockreadLock(); /** * Returns the lock used for writing. * * @return the lock used for writing. */ LockwriteLock(); }
ThreadPoolExecutor publicThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue)
A thread can also wake up without being notified, interrupted, or timing out, a so-called spurious wakeup. While this will rarely occur in practice, applications must guard against it by testing for the condition that should have caused the thread to be awakened, and continuing to wait if the condition is not satisfied. In other words, waits should always occur in loops, like this one: 线程也可以在没有被通知,中断或超时的情况下唤醒,即所谓的虚假唤醒。 虽然这在实践中很少发生,但应用程序必须通过测试应该导致线程被唤醒的条件来防范它,并且如果条件不满足则继续等待。 换句话说,等待应该总是出现在循环中
/** * Waits at most {@code millis} milliseconds for this thread to * die. A timeout of {@code 0} means to wait forever. * * <p> This implementation uses a loop of {@code this.wait} calls * conditioned on {@code this.isAlive}. As a thread terminates the * {@code this.notifyAll} method is invoked. It is recommended that * applications not use {@code wait}, {@code notify}, or * {@code notifyAll} on {@code Thread} instances. * * @param millis * the time to wait in milliseconds * * @throws IllegalArgumentException * if the value of {@code millis} is negative * * @throws InterruptedException * if any thread has interrupted the current thread. The * <i>interrupted status</i> of the current thread is * cleared when this exception is thrown. */ publicfinalsynchronizedvoidjoin(long millis) throws InterruptedException { longbase= System.currentTimeMillis(); longnow=0;
if (millis < 0) { thrownewIllegalArgumentException("timeout value is negative"); }
if (millis == 0) { while (isAlive()) { wait(0); } } else { while (isAlive()) { longdelay= millis - now; if (delay <= 0) { break; } wait(delay); now = System.currentTimeMillis() - base; } } }
/** Marker to indicate a node is waiting in shared mode */ staticfinalNodeSHARED=newNode(); /** Marker to indicate a node is waiting in exclusive mode */ staticfinalNodeEXCLUSIVE=null;
2、else 分支,如果尾节点不为 null, node.prev = t ,也就是将当前尾节点设置为待入队节点的前置节点。然后又是利用 CAS 操作,将待入队的节点设置为队列的尾节点,如果 CAS 返回 false,表示未设置成功,继续循环设置,直到设置成功,接着将之前的尾节点(也就是倒数第二个节点)的 next 属性设置为当前尾节点,对应 t.next = node 语句,然后返回当前尾节点,退出循环。
set hive.auto.convert.join=true; set hive.mapjoin.smalltable.filesize=300000000; set hive.auto.convert.join.noconditionaltask=true; set hive.auto.convert.join.noconditionaltask.size=300000000;
使用mapjoin时,会先执行一个本地任务(mapreduce local task)将小表转成hashtable并序列化为文件再压缩,随后这些hashtable文件会被上传到hadoop缓存,提供给各个mapjoin使用。这里有三个参数我们需要注意:
--是否自动转换为mapjoin set hive.auto.convert.join =true; --小表的最大文件大小,默认为25000000,即25M set hive.mapjoin.smalltable.filesize =25000000; --是否将多个mapjoin合并为一个 set hive.auto.convert.join.noconditionaltask =true; --多个mapjoin转换为1个时,所有小表的文件大小总和的最大值。 set hive.auto.convert.join.noconditionaltask.size =10000000;
hive的join 有一种优化的方式:map join
但是,使用这种优化的时候要小心一点,先说一下优化配置的参数:
localtask max memory usage: 本地任务内存百分比
1 2 3 4
set hive.optimize.correlation=true set hive.auto.convert.join=true set hive.mapjoin.localtask.max.memory.usage=0.99 hive.mapjoin.localtask.max.memory.usage