同步不适用于计数器？

Question

我编写了一个转换器，可以将一种文件格式（例如 "csv"）转换为另一种文件格式（例如 "json"）——对于小文件一切正常。对于较大的文件，我在 HashMap<Integer,String> 中使用的 int lineCounter 似乎 "jump"。 HashMap的第一个条目表示行号，HashMap的第二个条目表示数据。

源 CSV 看起来像这样（还有大约 3600 个条目）：

_id,actor.displayName,actor.id,actor.objectType,generator.displayName,generator.id,generator.objectType,generator.url,object.displayName,object.id,object.objectType,provider.displayName,provider.id,provider.inquiryPhase,provider.objectType,provider.url,published,publishedClient,publishedServer,target.displayName,target.id,target.inquiryPhase,target.objectType,verb
,BasKolloeffel(UT),BasKolloeffel(UT)@5485a7050ac61b1339a4da09,person,LochemC,5485a7050ac61b1339a4da09,ils,http://graasp.eu/spaces/5485a7050ac61b1339a4da09,LochemC,5485a7050ac61b1339a4da09,ils,LochemC,5485a7050ac61b1339a4da09,ils,ils,http://graasp.eu/spaces/5485a7050ac61b1339a4da09,2014-12-08T13:40:45.409Z,2014-12-08T13:40:45.409Z,,Orientation,5485a7050ac61b1339a4da0e,Orientation,phase,access

结果JSON：

{
    "actor" : {
        "displayName" : "BasKolloeffel(UT)",
        "id" : "BasKolloeffel(UT)@5485a7050ac61b1339a4da09",
        "objectType" : "person"
    },
    "generator" : {
        "displayName" : "LochemC",
        "id" : "5485a7050ac61b1339a4da09",
        "objectType" : "ils",
        "url" : "http://graasp.eu/spaces/5485a7050ac61b1339a4da09"
    },
    "object" : {
        "displayName" : "LochemC",
        "id" : "5485a7050ac61b1339a4da09",
        "objectType" : "ils"
    },
    "provider" : {
        "displayName" : "LochemC",
        "id" : "5485a7050ac61b1339a4da09",
        "inquiryPhase" : "ils",
        "objectType" : "ils",
        "url" : "http://graasp.eu/spaces/5485a7050ac61b1339a4da09"
    },
        "published" : "2014-12-08T13:40:45.409Z",
        "publishedClient" : "2014-12-08T13:40:45.409Z",
    "target" : {
        "displayName" : "Orientation",
        "id" : "5485a7050ac61b1339a4da0e",
        "inquiryPhase" : "Orientation",
        "objectType" : "phase"
    },
    "verb" : "access"
}

阅读了大约 2000 条转换后的 csv 行后，JSON-结构变得混乱并开始看起来像这样：

{
},
"displayName" : "Vogel1",
"actor" : {
    "objectType" : "person"
    "id" : "Vogel1@5485a7050ac61b1339a4da09",
"generator" : {
},
    "id" : "b0e88042-47ec-4bbb-e419-f997020956bc",
    "displayName" : "questioningscratchpad",
    "url" : "http://go-lab.gw.utwente.nl/experiments/2014-12-lochem/questioning_v1/tools/questioning/src/main/webapp/questioning_relative_density.xml"
    "objectType" : "application",
"object" : {
},
    "objectType" : "application"
    "id" : "b0e88042-47ec-4bbb-e419-f997020956bc",
"provider" : {
},
    "id" : "5485a7050ac61b1339a4da09",
    "displayName" : "LochemC",
    "objectType" : "ils",
    "inquiryPhase" : "Conceptualisation",
},
    "url" : "http://graasp.eu/spaces/5485a7050ac61b1339a4da09"
    "publishedClient" : "2014-12-16T10:27:55.097Z",
    "published" : "2014-12-16T10:27:55.097Z",
    "displayName" : "unnamedquestions",
"target" : {
    "objectType" : "questions"
    "id" : "b911e97c-cb7f-4cec-ab99-440aecb029f5",
    "verb" : "access"
}

我已经调试了 HashMap 的 lineCounter（上 50%，因为之前一切正常），我在使用 System.out.println(this.lineCounter); 时得到以下信息：

我希望对于 csv 文件中的每个标记，lineCounter 只会增加一 (+1)，而不是增加 30 或类似的值。我认为这是一个同步问题，但即使是同步块也不能解决这个问题。

我正在使用这 2 种方法来修改 HashMap：

public synchronized void putLineToMap(int lineCounter, String content) {
    this.lineMap.put(lineCounter, content);
    ++this.lineCounter;
}

public synchronized String replaceToken(String previousLine, String token, String replace) {
    --this.lineCounter;
    previousLine = this.readerSaver.replaceLast(previousLine, token, replace);
    return previousLine;
}

...为了完整起见，我将添加 replaceLast 方法：

/*
 * @param   string      The String-Object to be changed.
 * @param   substring   The String-Object-Part to be replaced.
 * @param   replacement The String-Object which should replace the substring.
 * @return                  The (changed) String-Object.
 */
public String replaceLast(String string, String substring, String replacement) {
    int index = string.lastIndexOf(substring);
    if (index == -1) {
        return string;
    }
    return string.substring(0, index) + replacement
            + string.substring(index + substring.length());
}

谁能帮帮我？

** 编辑 ** 这是我用来将 CSV 转换为 JSON.

的方法

/**
 * A utility function which gathers the data from a csv file to store it in
 * a for this fitting architecture. Afterwards the data will be processed
 * and stored in a valid JSON format.
 *
 * @param headerLineData The header of the CSV.
 * @param rowLineData ALL rows of the CSV with separator symbols between the
 * lines.
 * @param filepath The filepath where the JSON file should be saved to.
 * @param entries If a config file is used, one can find the header data to
 * parse in this hashset. If entries is null, the config file won't be used.
 */
public void createJSON(String headerLineData, String rowLineData, String filepath, TreeSet<String> entries) {

    try {
        String[] headerSeparatedData = headerLineData.split(",");

        //??? @flagLastValueLine
        boolean flagLastValueLine = false;
        //count the number of value rows.
        this.scan = new Scanner(rowLineData);
        while (this.scan.hasNext()) {
            this.scan.nextLine();
            ++this.numberLines;
        }

        //resetting the scanner
        this.scan = new Scanner(rowLineData);

        while (this.scan.hasNext()) {
            //Read first value row.
            String nextLine = this.scan.nextLine();
            //Split value-row.
            String[] rowData = nextLine.split(",");

            //For the first iteration - start tag
            if (this.whileLoopPassCounter == 0) {
                putLineToMap(this.lineCounter, "[");
            }

            //Debug
            if (this.lineCounter >= 65536) {
                System.out.println(this.lineCounter);
            }

            //Depth of current header node -> actor.id.ID -- Depth = 2
            int depthHeaderLevel = 0;

            //should keep in mind which key (without value-node) was last.
            Set<String> rememberKeyNode = new TreeSet<String>();

            //Iterate through all elements of the header row. If 4x "," = 5 Elements.
            for (int i = 0; i < headerSeparatedData.length; i++) {
                if (i == 0) {
                    putLineToMap(this.lineCounter, "{");
                }

                //proof whether the config file is used. Only parse the header data in this hashset.
                if (entries != null) {
                    Iterator<String> it = entries.iterator();
                    int size = entries.size();
                    int newSize = 0;
                    while (it.hasNext()) {
                        String key = it.next();
                        if (!headerSeparatedData[i].contains(key)) {
                            ++newSize;
                        }
                    }
                    if (newSize == size) {
                        continue;
                    }

                }

                //is rowData empty? Then jump to the next condition.
                if (!rowData[i].equals("")) {
                    //get all single elements of the header and split them again. case: have sub-elements.
                    if (headerSeparatedData[i].contains(".")) {
                        String[] headerSeparatedLevelData = headerSeparatedData[i].split("\.");
                        depthHeaderLevel = headerSeparatedLevelData.length;
                        //Iterate through the depth of Header Level
                        for (int k = 0; k < depthHeaderLevel; k++) {
                            //First case: headerlevel does NOT contain any direct value successor.
                            if (k <= depthHeaderLevel - 2 && !rememberKeyNode.contains(headerSeparatedLevelData[k])) {
                                //special case: does the last row contain a "," from the deepest level?
                                //if yes: set close-tag and begin a new block after this one.
                                int lines = 0;
                                synchronized (getClass()) {
                                    lines = this.lineCounter - 1;
                                }

                                String previousLine = this.lineMap.get(lines);
                                if (previousLine.contains(",") && this.flagSingleNode == false) {
                                    //decrement counter to edit the last line.
                                    previousLine = replaceToken(previousLine, ",", "");

                                    //edit last line and increment counter.
                                    putLineToMap(this.lineCounter, previousLine);
                                    //set close tag.
                                    putLineToMap(this.lineCounter, "},");
                                    //if one goes in an other level than the first: remove flag / set false.
                                    flagLastValueLine = false;
                                    /**
                                     * GEPFUSCHT!
                                     */
                                    //case: last row contains "},"
                                    int lines2 = 0;
                                    synchronized (getClass()) {
                                        lines2 = this.lineCounter - 1;
                                    }
                                    String prevLine = this.lineMap.get(lines2);
                                    if (prevLine.contains("},") && k == 0) {
                                        //decrement counter to remove the last line.
                                        prevLine = replaceToken(prevLine, "},", "}");

                                        synchronized (getClass()) {
                                            this.lineMap.remove(this.lineCounter);
                                        }

                                        //notice the depth of the header level.
                                        for (int j = 0; j < depthHeaderLevel - 1; j++) {
                                            //if end node not reached, just put "}"
                                            if (j < depthHeaderLevel - 2) {
                                                putLineToMap(this.lineCounter, "}");

                                            } //if end nose is reached, set close tag.
                                            else {
                                                putLineToMap(this.lineCounter, "},");

                                            }
                                        }
                                    }
                                }
                                rememberKeyNode.add(headerSeparatedLevelData[k]);
                                putLineToMap(this.lineCounter, "\"" + headerSeparatedLevelData[k] + "\" : {");

                                this.flagSingleNode = false;
                            } //second case: headerlevel has a text node as his successor.
                            else if (!rememberKeyNode.contains(headerSeparatedLevelData[k])) {
                                putLineToMap(this.lineCounter, "\"" + headerSeparatedLevelData[k] + "\" : " + "\"" + rowData[i] + "\",");

                                //if one is in the last level, flag should be set.
                                flagLastValueLine = true;
                            }
                        }
                    } //Header does not contain "." - Separation not necessary.
                    else {
                        //set the depthHeaderLevel to one, because there is only one element at the header.
                        depthHeaderLevel = 1;
                        if (!rememberKeyNode.contains(headerSeparatedData[i])) {
                            //if last line ends on ",", replace through "},"
                            int line = 0;
                            synchronized (getClass()) {
                                line = this.lineCounter - 1;
                            }
                            String prevLine = this.lineMap.get(line);
                            if (prevLine.contains(",") && !this.flagSingleNode) {
                                //decrement counter
                                prevLine = replaceToken(prevLine, ",", "");
                                putLineToMap(this.lineCounter, prevLine);
                                putLineToMap(this.lineCounter, "},");
                            }
                            putLineToMap(this.lineCounter, "\"" + headerSeparatedData[i] + "\" : " + "\"" + rowData[i] + "\",");
                            //if one is in the last level, flag should be set.
                            flagLastValueLine = true;
                            rememberKeyNode.add(headerSeparatedData[i]);
                            this.flagSingleNode = true;
                        }
                    }
                }
                //Three things done here:
                //1. if the last line contains a "," - remove it.
                //2. dynamically add the close tags of a BLOCK
                //3. reset the TreeSet
                if (i == headerSeparatedData.length - 1) {
                    int line = 0;
                    synchronized (getClass()) {
                        line = this.lineCounter - 1;
                    }
                    String prevLine = this.lineMap.get(line);
                    if (prevLine.contains(",")) {
                        //decrement counter to edit the last line with ",".
                        prevLine = replaceToken(prevLine, ",", "");
                        putLineToMap(this.lineCounter, prevLine);
                    }
                    //dynamically add the close tags of a BLOCK.
                    for (int l = 0; l < depthHeaderLevel; l++) {
                        if (l == depthHeaderLevel - 1) {
                            putLineToMap(this.lineCounter, "},");
                        } else {
                            putLineToMap(this.lineCounter, "}");
                        }

                    }
                    rememberKeyNode = new TreeSet<String>();
                }
            }
            this.whileLoopPassCounter++;

            if (this.whileLoopPassCounter == this.numberLines) {
                //remove last "," - if there is one.
                String prevLine = this.lineMap.get(this.lineCounter - 1);
                if (prevLine.contains(",")) {
                    //decrement counter to edit the last line.
                    prevLine = replaceToken(prevLine, ",", "");
                    putLineToMap(this.lineCounter, prevLine);
                }

                //close tag
                putLineToMap(this.lineCounter, "]");
            }
        }
        this.scan.close();
        saveResultToFile(this.lineMap, filepath);
    } catch (IOException ex) {
        Logger.getLogger(CSVtoJSONStructureHelper.class.getName()).log(Level.SEVERE, null, ex);
    }
}

Answer 1

我来猜一猜。

如果您的程序使用前几个数据点，那么我猜您的其余数据不那么干净。也许某些行在您的 csv 文件中的某些列中缺少值。我建议您检查从输入文件中读取的每一行。

a. read one line from file
b. check if the line is in the expected format
     b.1 tokenized the line
     b.2 check if the number of tokens (length) is equal to the expected value, if not discard the line or do some thing to salvage the incomplete data point.

Answer 2

您提到使用 HashMap 来保存您的 Map<Integer, String> 行号 + json 条目，但是 HashMap 不维护条目迭代的插入顺序。

如果需要保证迭代顺序与插入顺序相同，可以使用LinkedHashMap

同步不适用于计数器？

synchronization doesn't working with counter?

java

csv

json

synchronization