mlcp 将 csv 文件转换为 OBI 源

Question

我有以下挑战。我们有要使用 mlcp 加载到 MarkLogic 数据库中的 csv 文件。我们还想在加载期间将加载的行转换为 OBI 源，因此我们为此构建了一个转换函数。

现在我正在为转型而苦苦挣扎。如果没有转换，数据将按预期作为每行文档加载。

csv 示例：

voornaam,achternaam
hugo,koopmans
thijs,van ulden

变换-ambulance.xqy:

xquery version "1.0-ml";
module namespace rws = "http://marklogic.com/rws";

import module namespace source = "http://marklogic.com/solutions/obi/source" at "/ext/obi/lib/source-lib.xqy";

(: If the input document is XML, create an OBI source from it, with the value
 : specified in the input parameter. If the input document is not
 : XML, leave it as-is.
 :)
declare function rws:transform(
  $content as map:map,
  $context as map:map
) as map:map*
{
  let $attr-value := 
    (map:get($context, "transform_param"), "UNDEFINED")[1]
  let $the-doc := map:get($content, "value")
  return
    if (fn:empty($the-doc/element()))
    then $content
    else
      let $root := xdmp:unquote($the-doc/*)
      let $source-title := "ambulance source data"
      let $collection := 'ambulance'
      let $source-id := source:create-source($source-title, (),$root)      
      let $_ := xdmp:document-add-collections(concat("/marklogic.solutions.obi/source/", $source-id[1],".xml"), $collection)
      return (
        map:put($content, "value",
          $source-id[2]
        ), $content
      )
};

mlcp 命令：

mlcp.sh import \
 -host localhost \
 -port 27041 \
 -username admin \
 -password admin \
 -input_file_path ./sampledata/so-example.csv \
 -input_file_type delimited_text \
 -transform_module /transforms/transform-ambulance.xqy \
 -transform_namespace "http://marklogic.com/rws" \
 -mode local

mlcp 输出：

15/09/08 21:35:08 INFO contentpump.ContentPump: Hadoop library version: 2.6.0
15/09/08 21:35:08 INFO contentpump.LocalJobRunner: Content type: XML
15/09/08 21:35:08 INFO input.FileInputFormat: Total input paths to process : 1
15/09/08 21:35:10 WARN mapreduce.ContentWriter: XDMP-DOCROOTTEXT: xdmp:unquote(document{<root><voornaam>hugo</voornaam><achternaam>koopmans</achternaam></root>}) -- Invalid root text "hugokoopmans" at  line 1
15/09/08 21:35:10 WARN mapreduce.ContentWriter: XDMP-DOCROOTTEXT: xdmp:unquote(document{<root><voornaam>thijs</voornaam><achternaam>van ulden</achternaam></root>}) -- Invalid root text "thijsvan ulden" at  line 1
15/09/08 21:35:11 INFO contentpump.LocalJobRunner:  completed 100%
15/09/08 21:35:11 INFO contentpump.LocalJobRunner: com.marklogic.contentpump.ContentPumpStats: 
15/09/08 21:35:11 INFO contentpump.LocalJobRunner: ATTEMPTED_INPUT_RECORD_COUNT: 2
15/09/08 21:35:11 INFO contentpump.LocalJobRunner: SKIPPED_INPUT_RECORD_COUNT: 0
15/09/08 21:35:11 INFO contentpump.LocalJobRunner: Total execution time: 2 sec

我试过没有 xdmp:unquote() 但后来我遇到了一个强制文档节点 () 错误...

请指教...

Answer 1

好的，所以问题是我们需要将 $root 变量转换为 document-node()...

let $root := document {$the-doc/root}

问题已解决。

mlcp 将 csv 文件转换为 OBI 源

mlcp transform csv file into OBI sources

marklogic

marklogic-8

mlcp