php 脚本在 postgresql 上运行时重复自身

Question

我运行 php 脚本 select/insert 到 postgresql 数据库。

问题是，当我将脚本运行到 select 多于 (400000) 行时，

脚本重复插入相同的行！

我在 postgresql 日志中发现了这个错误：

2014-12-21 13:14:53 AST LOG:  checkpoints are occurring too frequently (7 seconds apart)
2014-12-21 13:14:53 AST HINT:  Consider increasing the configuration parameter "checkpoint_segments".

我将配置更改为：

- Checkpoints -
checkpoint_segments = 100       # in logfile segments, min 1, 16MB each
checkpoint_timeout = 1h     # range 30s-1h
checkpoint_completion_target = 0.9  # checkpoint target duration, 0.0 - 1.0
checkpoint_warning = 30s        # 0 disables

但问题依旧。谁能告诉我如何解决这个问题？

更新

我用一个准备查询开始我的脚本：

<?php
// * Connect to Specified Database
set_time_limit(0);
$dbconn = pg_connect("host=localhost port=5432 dbname=postgis user=postgres password=****");

    if (!$dbconn) { die("Error in connection: " . pg_last_error());}

// connected to Database
 else{

            //We prepare the PostgreSQL next messages query. At this stage it is sent off to the Database server.

                $nextMessageQuery = pg_prepare($dbconn, 'selectNextMessage',  "
                        WITH 
                            history AS (select id ,userid,assigning_date from userid_history 
                            where id= and assigning_date <),
                            accounttbl AS(select timestamp,userid,position from account
                            where timestamp > and position is not null
                            and timestamp between  and )

                        select  p.value1 ,value2.value3,p.value4, m.userid 
                        from (
                            SELECT min(next.timestamp)AS value3,next.userid,next.id  from(
                            select history.id,accounttbl.userid,accounttbl.timestamp,history.assigning_date,accounttbl.position 
                            from history 
                            inner join accounttbl
                            on(
                            accounttbl.userid=history.userid and 
                            accounttbl.timestamp > history.assigning_date and 
                                ( accounttbl.timestamp <(select min(assigning_date) from history h 
                                                    where h.id=history.id and
                                                    h.assigning_date>history.assigning_date) 
                                                or 
                                                    (select min(assigning_date) from history h 
                                                        where h.id=history.id and h.assigning_date>history.assigning_date)  is null
                                )
                            )
                            where timestamp >
                            )next GROUP BY next.userid,next.id 
                        ) value2
                        JOIN nmea m on  m.timestamp=value2.value3 and m.userid=value2.userid ,places p 
                        WHERE  m.position is not null and ST_DWithin(m.position,p.position,0.0217130577252428)order by ST_Distance(p.position,m.position)");


                    ?>

这只是为了向您展示我的查询有多么复杂。

然后 select #行，和根据许多比较语句

插入或更新到新的 table

所有内容都在一个复杂的脚本中。我必须全部 post 吗？

Answer 1

谢谢大家，

正如 Craig 提到的，数据本身存在错误，而不是数据加载。

实际上，我们select在某些记录中存在复制！

所以我用了：

SELECT DISTINCT on (time , id) time , id ,.... FROM table

php 脚本在 postgresql 上运行时重复自身

php script repeat itself while running on postgresql

php

postgresql