#
Jenkins遠程部署，一開始沒有任何頭緒，想了很多方案. 因為兩臺機器都是windows系統，所以想到publish over cifs, 但是這個網上資料太少，貌似只能內網使用。又想到了Jenkins 分布式構建，但是Jenkins構建的代碼和產物最后自動拷貝到主節點。而遠程機器其實是客戶方的機器，所以這個分布式構建并不適用。最后還是選定publish over ssh來實現遠程部署。
請注意：在進行遠程部署操作前，先要確保客戶機能ssh 登錄到遠程機器。如果不知道SSH怎么登陸，請參考http://blog.csdn.net/flyingshuai/article/details/72897692
1. 安裝publish over ssh 插件，安裝很簡單，在此不表。
2. 在Jenkins系統設置里找到Publish over SSH模塊
3. 用戶名/密碼方式登錄的，系統設置里設置如下：
4. 如果是證書登錄的，系統設置里設置如下：
5. Job設置，點擊增加構建后操作步驟，選擇send build artifacts over ssh, 設置如下：
6. 文件上傳到遠程服務器后，還有一些后續操作，比如，替換數據庫配置文件。可以把bat命令寫到一個批處理文件中，存到服務器上。Exec command填寫批處理文件的絕對路徑。如上圖所示。
關于bat腳本：
如果每次都需要替換同樣的文件，用copy /y 是無條件覆蓋，不會詢問。而xcopy可以實現批量拷貝文件和文件夾。如果文件較多可用此命令
注意腳本運行失敗，構建也會顯示藍色成功圖標，所以一定要打開控制臺輸出，看是否真的成功。
---------------------
作者：flyingshuai
來源：CSDN
版權聲明：本文為博主原創文章，轉載請附上博文鏈接！
摘要: 問題:I recently updated the configuration of one of my hudson builds. The build history is out of sync. Is there a way to clear my build history?Please and thank you回答1:If you click Manage Hudson / Relo...
閱讀全文
@Configuration
@DependsOn(value="cachingConnectionFactory")
public class JmsTemplateConfiguration {
@Value("${wechat.sendmessage.queue}")
private String queueName;
@Value("${wechat.sendmessage.topic}")
private String topicName;
@Value("${spring.jms.pub-sub-domain}")
private boolean isPubSubDomain;
/**
* 定義點對點隊列
* @return
*/
@Bean
public Queue queue() {
return new ActiveMQQueue(queueName);
}
/**
* 定義一個主題
* @return
*/
@Bean
public Topic topic() {
return new ActiveMQTopic(topicName);
}
private final ObjectProvider<DestinationResolver> destinationResolver;
private final ObjectProvider<MessageConverter> messageConverter;
private final CachingConnectionFactory cachingConnectionFactory;
@Autowired
public JmsTemplateConfiguration(ObjectProvider<DestinationResolver> destinationResolver,
ObjectProvider<MessageConverter> messageConverter,
CachingConnectionFactory cachingConnectionFactory) {
this.destinationResolver = destinationResolver;
this.messageConverter = messageConverter;
this.cachingConnectionFactory = cachingConnectionFactory;
}
/**
* 配置隊列生產者的JmsTemplate
* @return JmsTemplate
*/
@Bean(name="jmsQueueTemplate")
public JmsTemplate jmsQueueTemplate() {
//設置創建連接的工廠
//JmsTemplate jmsTemplate = new JmsTemplate(connectionFactory);
//優化連接工廠，這里應用緩存池 連接工廠就即可
JmsTemplate jmsTemplate = new JmsTemplate(cachingConnectionFactory);
//設置默認消費topic
//jmsTemplate.setDefaultDestination(topic());
//設置P2P隊列消息類型
jmsTemplate.setPubSubDomain(isPubSubDomain);
DestinationResolver destinationResolver = (DestinationResolver) this.destinationResolver.getIfUnique();
if (destinationResolver != null) {
jmsTemplate.setDestinationResolver(destinationResolver);
}
MessageConverter messageConverter = (MessageConverter) this.messageConverter.getIfUnique();
if (messageConverter != null) {
jmsTemplate.setMessageConverter(messageConverter);
}
//deliveryMode, priority, timeToLive 的開關，要生效，必須配置為true，默認false
jmsTemplate.setExplicitQosEnabled(true);
//DeliveryMode.NON_PERSISTENT=1:非持久 ; DeliveryMode.PERSISTENT=2:持久
//定義持久化后節點掛掉以后，重啟可以繼續消費.
jmsTemplate.setDeliveryMode(DeliveryMode.PERSISTENT);
//默認不開啟事務
System.out.println("默認是否開啟事務："+jmsTemplate.isSessionTransacted());
//如果不啟用事務，則會導致XA事務失效；
//作為生產者如果需要支持事務，則需要配置SessionTransacted為true
//jmsTemplate.setSessionTransacted(true);
//消息的應答方式，需要手動確認，此時SessionTransacted必須被設置為false，且為Session.CLIENT_ACKNOWLEDGE模式
//Session.AUTO_ACKNOWLEDGE 消息自動簽收
//Session.CLIENT_ACKNOWLEDGE 客戶端調用acknowledge方法手動簽收
//Session.DUPS_OK_ACKNOWLEDGE 不必必須簽收，消息可能會重復發送
jmsTemplate.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
return jmsTemplate;
}
/**
* 配置發布訂閱生產者的JmsTemplate
* @return JmsTemplate
*/
@Bean(name="jmsTopicTemplate")
public JmsTemplate jmsTopicTemplate() {
//設置創建連接的工廠
//JmsTemplate jmsTemplate = new JmsTemplate(connectionFactory);
//優化連接工廠，這里應用緩存池 連接工廠就即可
JmsTemplate jmsTemplate = new JmsTemplate(cachingConnectionFactory);
//設置默認消費topic
//jmsTemplate.setDefaultDestination(topic());
//設置發布訂閱消息類型
jmsTemplate.setPubSubDomain(isPubSubDomain);
//deliveryMode, priority, timeToLive 的開關，要生效，必須配置為true，默認false
jmsTemplate.setExplicitQosEnabled(true);
//DeliveryMode.NON_PERSISTENT=1:非持久 ; DeliveryMode.PERSISTENT=2:持久
jmsTemplate.setDeliveryMode(DeliveryMode.PERSISTENT);
//默認不開啟事務
System.out.println("是否開啟事務"+jmsTemplate.isSessionTransacted());
//如果session帶有事務，并且事務成功提交，則消息被自動簽收。如果事務回滾，則消息會被再次傳送。
//jmsTemplate.setSessionTransacted(true);
//不帶事務的session的簽收方式，取決于session的配置。
//默認消息確認方式為1，即AUTO_ACKNOWLEDGE
System.out.println("是否消息確認方式"+jmsTemplate.getSessionAcknowledgeMode());
//消息的應答方式，需要手動確認，此時SessionTransacted必須被設置為false，且為Session.CLIENT_ACKNOWLEDGE模式
//Session.AUTO_ACKNOWLEDGE 消息自動簽收
//Session.CLIENT_ACKNOWLEDGE 客戶端調用acknowledge方法手動簽收
//Session.DUPS_OK_ACKNOWLEDGE 不必必須簽收，消息可能會重復發送
jmsTemplate.setSessionAcknowledgeMode(Session.CLIENT_ACKNOWLEDGE);
return jmsTemplate;
}
}
Why Enterprise Integration Patterns?
Enterprise integration is too complex to be solved with a simple 'cookbook' approach. Instead, patterns can provide guidance by documenting the kind of experience that usually lives only in architects' heads: they are accepted solutions to recurring problems within a given context. Patterns are abstract enough to apply to most integration technologies, but specific enough to provide hands-on guidance to designers and architects. Patterns also provide a vocabulary for developers to efficiently describe their solution.
Patterns are not 'invented'; they are harvested from repeated use in practice. If you have built integration solutions, it is likely that you have used some of these patterns, maybe in slight variations and maybe calling them by a different name. The purpose of this site is not to "invent" new approaches, but to present a coherent collection of relevant and proven patterns, which in total form an integration pattern language.
Despite the 700+ pages, our book covers only a fraction of patterns (and the problems to be solved) in the integration space. The current patterns focus on Messaging, which forms the basis of most other integration patterns. We have started to harvest more patterns but are realizing (once again) how much work documenting these patterns really is. So please stay tuned.
Messaging Patterns
We have documented 65 messaging patterns, organized as follows:
https://www.enterpriseintegrationpatterns.com/patterns/messaging/index.html
A detailed step-by-step tutorial on how to connect to a JMS broker using a Spring Integration Gateway and Spring Boot.
A detailed step-by-step tutorial on how to connect to Apache ActiveMQ Artemis using Spring JMS and Spring Boot.
A detailed step-by-step tutorial on how to publish/subscribe to a JMS topic using Spring JMS and Spring Boot.
A detailed step-by-step tutorial on how to connect to an ActiveMQ JMS broker using Spring Integration and Spring Boot.
A detailed step-by-step tutorial on how a Spring JMS listener works in combination with Spring Boot.
A detailed step-by-step tutorial on how to use JmsTemplate in combination with Spring JMS and Spring Boot.
A detailed step-by-step tutorial on how to implement a message selector using Spring JMS and Spring Boot.
A detailed step-by-step tutorial on how to implement a message converter using Spring JMS and Spring Boot.
A detailed step-by-step tutorial on how to use a Spring Boot admin UI to manage Spring Batch jobs.
A detailed step-by-step tutorial on how to implement a Spring Batch Tasklet using Spring Boot.
A detailed step-by-step tutorial on how to implement a Hello World Spring Batch job using Spring Boot.
This time I decided to play a little bit with Spring Integration Java DSL. Which has been merged directly into Spring Integration Core 5.0, which is smart and obvious move because:
- Everyone starting the new Spring projects based on Java Config uses that
- SI Java DSL enables you to use new powerfull Java 8 features like Lambdas
- You can build your flow using the Builder pattern based on IntegrationFlowBuilder
Let's take a look on the samples howto use that based on ActiveMQ JMS.
https://bitbucket.org/tomask79/spring-integration-java-dsl/src/master/
SPRING BATCH remote chunking模式下，如果要同一時間處理多個文件，按DEMO的默認配置，是會報錯的，這是由于多個文件的處理的MASTER方，是用同一個QUEUE名，這樣SLAVE中處理多個JOB INSTANCE時，會返回不同的JOB-INSTANCE-ID，導致報錯。
這時需更改SPRING BATCH使用SPRING INTEGRATION的模式中的GATEWAY組件。
GATEWAY組件是工作在REQUEST/RESPONSE模式下，即發一個MESSAGE到某一QUEUE時，要從REPLY QUEUE等到CONSUMER返回結果時，才往下繼續。
OUTBOUND GATEWAY：從某一CHANNEL獲取MESSAGE，發往REQUEST QUEUE，從REPLY QUEUE等到CONSUMER返回結果，將此MESSAGE發往下一CHANNEL。
INBOUND GATEWAY：從某一QUEUE獲取MESSAGE，發往某一REQUEST CHANNEL，從REPLY CHANNEL等到返回結果，將此MESSAGE發往下一QUEUE。
詳情參見此文：
https://blog.csdn.net/alexlau8/article/details/78056064。
<!-- Master jms -->
<int:channel id="MasterRequestChannel">
<int:dispatcher task-executor="RequestPublishExecutor"/>
</int:channel>
<task:executor id="RequestPublishExecutor" pool-size="5-10" queue-capacity="0"/>
<!-- <int-jms:outbound-channel-adapter
connection-factory="connectionFactory"
destination-name="RequestQueue"
channel="MasterRequestChannel"/> -->
<int:channel id="MasterReplyChannel"/>
<!-- <int-jms:message-driven-channel-adapter
connection-factory="connectionFactory"
destination-name="ReplyQueue"
channel="MasterReplyChannel"/> -->
<int-jms:outbound-gateway
connection-factory="connectionFactory"
correlation-key="JMSCorrelationID"
request-channel="MasterRequestChannel"
request-destination-name="RequestQueue"
receive-timeout="30000"
reply-channel="MasterReplyChannel"
reply-destination-name="ReplyQueue"
async="true">
<int-jms:reply-listener />
</int-jms:outbound-gateway>
<!-- Slave jms -->
<int:channel id="SlaveRequestChannel"/>
<!-- <int-jms:message-driven-channel-adapter
connection-factory="connectionFactory"
destination-name="RequestQueue"
channel="SlaveRequestChannel"/> -->
<int:channel id="SlaveReplyChannel"/>
<!-- <int-jms:outbound-channel-adapter
connection-factory="connectionFactory"
destination-name="ReplyQueue"
channel="SlaveReplyChannel"/> -->
<int-jms:inbound-gateway
connection-factory="connectionFactory"
correlation-key="JMSCorrelationID"
request-channel="SlaveRequestChannel"
request-destination-name="RequestQueue"
reply-channel="SlaveReplyChannel"
default-reply-queue-name="ReplyQueue"/>
MASTER配置
package com.paul.testspringbatch.config.master;
import javax.jms.ConnectionFactory;
import org.springframework.beans.factory.config.CustomScopeConfigurer;
//import org.springframework.batch.core.configuration.annotation.StepScope;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.context.annotation.Scope;
import org.springframework.context.support.SimpleThreadScope;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.channel.QueueChannel;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.jms.JmsOutboundGateway;
import com.paul.testspringbatch.common.constant.IntegrationConstant;
@Configuration
@EnableIntegration
@Profile("batch-master")
public class IntegrationMasterConfiguration {
// @Value("${broker.url}")
// private String brokerUrl;
// @Bean
// public ActiveMQConnectionFactory connectionFactory() {
// ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory();
// connectionFactory.setBrokerURL(this.brokerUrl);
// connectionFactory.setTrustAllPackages(true);
// return connectionFactory;
// }
/*
* Configure outbound flow (requests going to workers)
*/
@Bean
// @Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public DirectChannel requests() {
return new DirectChannel();
}
// @Bean
// public IntegrationFlow outboundFlow(ConnectionFactory connectionFactory) {
// return IntegrationFlows
// .from(requests())
// .handle(Jms.outboundAdapter(connectionFactory).destination(IntegrationConstant.MASTER_REQUEST_DESTINATION))
// .get();
// }
@Bean
public CustomScopeConfigurer customScopeConfigurer() {
CustomScopeConfigurer customScopeConfigurer = new CustomScopeConfigurer();
customScopeConfigurer.addScope("thread", new SimpleThreadScope());
return customScopeConfigurer;
}
// @Bean
// public static BeanFactoryPostProcessor beanFactoryPostProcessor() {
// return new BeanFactoryPostProcessor() {
//
// @Override
// public void postProcessBeanFactory(ConfigurableListableBeanFactory beanFactory) throws BeansException {
// beanFactory.registerScope("thread", new SimpleThreadScope());
// }
// };
// }
/*
* Configure inbound flow (replies coming from workers)
*/
@Bean
@Scope(value = "thread"/* , proxyMode = ScopedProxyMode.NO */)
// @Scope(ConfigurableBeanFactory.SCOPE_PROTOTYPE)
public QueueChannel replies() {
return new QueueChannel();
}
// @Bean
// public IntegrationFlow inboundFlow(ConnectionFactory connectionFactory) {
// return IntegrationFlows
// .from(Jms.messageDrivenChannelAdapter(connectionFactory).destination(IntegrationConstant.MASTER_REPLY_DESTINATION))
// .channel(replies())
// .get();
// }
@Bean
public JmsOutboundGateway jmsOutboundGateway(ConnectionFactory connectionFactory) {
JmsOutboundGateway jmsOutboundGateway = new JmsOutboundGateway();
jmsOutboundGateway.setConnectionFactory(connectionFactory);
jmsOutboundGateway.setRequestDestinationName(IntegrationConstant.MASTER_REQUEST_DESTINATION);//2. send the message to this destination
jmsOutboundGateway.setRequiresReply(true);
jmsOutboundGateway.setCorrelationKey(IntegrationConstant.JMS_CORRELATION_KEY);//3. let the broker filter the message
jmsOutboundGateway.setAsync(true);//must be async, so that JMS_CORRELATION_KEY work
jmsOutboundGateway.setUseReplyContainer(true);
jmsOutboundGateway.setReplyDestinationName(IntegrationConstant.MASTER_REPLY_DESTINATION);//4. waiting the response from this destination
jmsOutboundGateway.setReceiveTimeout(30_000);
return jmsOutboundGateway;
}
@Bean
public IntegrationFlow jmsOutboundGatewayFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows
.from(requests())//1. receive message from this channel
.handle(jmsOutboundGateway(connectionFactory))
.channel(replies())//5. send back the response to this channel
.get();
}
}
SLAVE配置：
package com.paul.testspringbatch.config.slave;
import javax.jms.ConnectionFactory;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Profile;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.jms.dsl.Jms;
import com.paul.testspringbatch.common.constant.IntegrationConstant;
@Configuration
@EnableIntegration
@Profile("batch-slave")
public class IntegrationSlaveConfiguration {
/*
* Configure inbound flow (requests coming from the master)
*/
@Bean
public DirectChannel requests() {
return new DirectChannel();
}
// @Bean
// public IntegrationFlow inboundFlow(ConnectionFactory connectionFactory) {
// return IntegrationFlows
// .from(Jms.messageDrivenChannelAdapter(connectionFactory).destination("requests"))
// .channel(requests())
// .get();
// }
/*
* Configure outbound flow (replies going to the master)
*/
@Bean
public DirectChannel replies() {
return new DirectChannel();
}
// @Bean
// public IntegrationFlow outboundFlow(ConnectionFactory connectionFactory) {
// return IntegrationFlows
// .from(replies())
// .handle(Jms.outboundAdapter(connectionFactory).destination("replies"))
// .get();
// }
@Bean
public IntegrationFlow inboundGatewayFlow(ConnectionFactory connectionFactory) {
return IntegrationFlows
.from(Jms
.inboundGateway(connectionFactory)
.destination(IntegrationConstant.SLAVE_HANDLE_MASTER_REQUEST_DESTINATION)//1. receive message from this channel.
.correlationKey(IntegrationConstant.JMS_CORRELATION_KEY)//2. let the broker filter the message
.requestChannel(requests())//3. send the message to this channel
.replyChannel(replies())//4. waitting the result from this channel
.defaultReplyQueueName(IntegrationConstant.SLAVE_RETURN_RESULT_DESTINATION)//5.send back the result to this destination to the master.
)
.get();
}
}
在SPRING BATCH中，通常ROUTER是針對STEP的，但是如果在一個STEP中有多個WRITER，每個WRITER是寫不同文件的，因此需要一個STEP內的ROUTER，以便能ROUTE到不同的WRITER中。
https://gist.github.com/benas/bfe2be7386b99ce496425fac9ff35fb8
在SPRING BATCH REMOTE CHUNKING的模式下：
SPRING BATCH 讀文件時，是按一行一行來讀取數據，再按CHUNKSIZE提交到REMOTE操作，有時要整合當前行和下幾行，再決定CHUNKSIZE，以便相關的數據能在遠程同一個PROCESSOR中按順序進行處理，因為相關的數據被拆成幾個CHUNK來處理的話，就有可能不按順序來處理。這樣就需要動態調整CHUNKSIZE。
參照如下：
https://stackoverflow.com/questions/37390602/spring-batch-custom-completion-policy-for-dynamic-chunk-size
并結合SingleItemPeekableItemReader（裝飾者，允許查看下一條數據，真正的操作委托給代理）。