Navigation

    • Register
    • Login
    • Search
    • Categories
    • Recent
    • Tags
    • Popular
    • Users
    • Groups

    UNSOLVED mysql全量迁移至clickhouse报错

    问答区
    mysql clickhouse 全量迁移
    1
    2
    234
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      Time. last edited by

      CloudCanal 运行版本:2.1.0.14
      宿主机操作系统版本:Centos7
      数据库源端、目标端类型:MYSQL 5.7.37 ==> clickhouse 21.3.20.1
      问题描述:源数据库连接测试正常,任务全量迁移阶段异常,部分表数据(数据量少)可迁移成功,大表数据迁移报错。
      异常堆栈信息:
      
      com.clougence.cloudcanal.base.service.task.exception.runtime.FullScanException: null.chs_datacenter_new.chs_core_basic_settlement_detail_hz_2021_4 full extract occur an exception,msg:DataSourceDisableException: 
      	at com.clougence.cloudcanal.base.service.task.parser.full.RdbSinglePkPageScanner.scanTable(RdbSinglePkPageScanner.java:189)
      	at com.clougence.cloudcanal.mysql.worker.reader.full.MySQLTableFullProducer.produce(MySQLTableFullProducer.java:94)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: com.alibaba.druid.pool.DataSourceDisableException
      	at com.alibaba.druid.pool.DruidDataSource.close(DruidDataSource.java:2105)
      	at com.clougence.cloudcanal.base.service.dsfactory.RdbDsFactory.lambda$apply$0(RdbDsFactory.java:91)
      	at com.clougence.cloudcanal.base.plugin.api.dsfactory.RefDsHolder.close(RefDsHolder.java:24)
      	at com.clougence.cloudcanal.task.extractor.full.RdbFullExtractor.stop(RdbFullExtractor.java:133)
      	at com.clougence.cloudcanal.task.service.impl.CanalFullServiceImpl.stop(CanalFullServiceImpl.java:90)
      	at com.clougence.cloudcanal.task.DataTaskStarter.stop(DataTaskStarter.java:202)
      	... 1 more
      

      MYSQL端报错信息:

      2022-02-09T01:35:38.788323Z 251 [Note] Aborted connection 251 to db: 'chs_datacenter_new' user: 'cloudcanal' host: '192.168.100.11' (Got an error writing communication packets)
      
      T 1 Reply Last reply Reply Quote 0
      • T
        Time. @Time. last edited by

        @time 在 mysql全量迁移至clickhouse报错 中说:

        CloudCanal 运行版本:2.1.0.14
        宿主机操作系统版本:Centos7
        数据库源端、目标端类型:MYSQL 5.7.37 ==> clickhouse 21.3.20.1
        问题描述:源数据库连接测试正常,任务全量迁移阶段异常,部分表数据(数据量少)可迁移成功,大表数据迁移报错。
        异常堆栈信息:
        
        com.clougence.cloudcanal.base.service.task.exception.runtime.FullScanException: null.chs_datacenter_new.chs_core_basic_settlement_detail_hz_2021_4 full extract occur an exception,msg:DataSourceDisableException: 
        	at com.clougence.cloudcanal.base.service.task.parser.full.RdbSinglePkPageScanner.scanTable(RdbSinglePkPageScanner.java:189)
        	at com.clougence.cloudcanal.mysql.worker.reader.full.MySQLTableFullProducer.produce(MySQLTableFullProducer.java:94)
        	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
        	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
        	at java.lang.Thread.run(Thread.java:748)
        Caused by: com.alibaba.druid.pool.DataSourceDisableException
        	at com.alibaba.druid.pool.DruidDataSource.close(DruidDataSource.java:2105)
        	at com.clougence.cloudcanal.base.service.dsfactory.RdbDsFactory.lambda$apply$0(RdbDsFactory.java:91)
        	at com.clougence.cloudcanal.base.plugin.api.dsfactory.RefDsHolder.close(RefDsHolder.java:24)
        	at com.clougence.cloudcanal.task.extractor.full.RdbFullExtractor.stop(RdbFullExtractor.java:133)
        	at com.clougence.cloudcanal.task.service.impl.CanalFullServiceImpl.stop(CanalFullServiceImpl.java:90)
        	at com.clougence.cloudcanal.task.DataTaskStarter.stop(DataTaskStarter.java:202)
        	... 1 more
        

        MYSQL端报错信息:

        2022-02-09T01:35:38.788323Z 251 [Note] Aborted connection 251 to db: 'chs_datacenter_new' user: 'cloudcanal' host: '192.168.100.11' (Got an error writing communication packets)
        

        2022-02-09 补充说明
        经查看clickhouse日志发现,是写入clickhouse报错了,问题日志:

        2022.02.09 14:59:47.821263 [ 263 ] {452dc7bb-f34d-4a28-9801-f2dc2b2a66d0} <Error> executeQuery: Code: 27. DB::ParsingException: Cannot parse input: expected '\t' before
        
        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        Copyright © 2020 ClouGence, Inc.备案号:浙ICP备20007605号-2