在加载程序中对大数据集执行的查询是 slow/causing UI 到 freeze/stutter
Query executing on large data set in loader is slow/causing UI to freeze/stutter
我有一个查询从 table 获取 MAX(time)，GPS 位置包含 300,000 个条目。我有时间索引以及 USER,TIME 索引。
我访问加载器中的位置，每次将新位置插入 table 时都会收到通知，并将刷新加载器信息，从而调用查询
但我的问题是，几乎每次发生这种情况时，UI 都会锁定，直到查询完成。
我知道这个是因为我在提供者中添加了一个Log.d来显示每次执行查询需要多少毫秒。有时查询将在 30 毫秒（ish）内执行，但大多数情况下需要 2000-4000 毫秒，有时甚至高达 6000 毫秒。在此期间 UI 将停止响应。
我有三个问题，
- 加载器的使用不是应该以这种方式解耦 UI 的减速，这样长查询就不会导致 UI 在执行时锁定吗？
- 是否有任何事情会导致查询等待 "lock" 而导致它确实延迟执行，从而使查询花费更长的时间？
- 为什么查询的执行速度似乎有时变化很快，但大多数时候却很慢。
我已尝试将提供程序中的查询简化为以下内容，仅用于测试：
queryBuilder = new SQLiteQueryBuilder();
tables = PositionEntry.TABLE_NAME;
queryBuilder.setTables(tables);
projectionMap = new HashMap();
projectionMap.put(PositionEntry.COLUMN_USER, PositionEntry.COLUMN_USER);
projectionMap.put(PositionEntry.COLUMN_TIME,"MAX("+PositionEntry.COLUMN_TIME + ") AS " +PositionEntry.COLUMN_TIME );
queryBuilder.setProjectionMap(projectionMap);
long start1 = System.currentTimeMillis();
retCursor = queryBuilder.query(mOpenHelper.getReadableDatabase(), null, null, null, PositionEntry.COLUMN_USER, null, null, null);
Log.d(LOG_TAG,"Sub Query took " + (System.currentTimeMillis() - start1) + "ms to complete Rows:" + retCursor.getCount());
我返回了 3 行，这是正确的，并且我根据每个用户的上次时间获得了他们的最后位置。但是查询每次执行需要2秒左右。
//编辑：修正了一个代码拼写错误
也许考虑有一个单行table，当一行被插入主table.
时，最大时间通过触发器自动更新
假设一个名为 lastentry 的 table 具有列 lastentry_id（INTEGER 但不是 rowid 的别名）和 lastentry_time（最大时间为整数）并且 master/main table 被称为 tracker id 列和 time 列（加上其他）
然后是下面
CREATE TRIGGER trg_update_lastentry
AFTER INSERT ON tracker
BEGIN
UPDATE lastentry
SET
lastentry_id = new._id,
lastentry_time = new.tracker_time
WHERE lastentry_time < new.tracker_time
;
END
将更新 lastentry table 中的单行（即更改 lastentry_time 列和 lastentry_id 列（参考相应 tracker 的 ID table行)) 如果trackertable中的时间大于lastnetry[=44中的时间=] table.
因此您可以提取最大时间而无需查询 300,000 行主（跟踪器）table。
Testing/Example
DBHelper.java :-
public class DBHelper extends SQLiteOpenHelper {
public static final String DBNAME = "traking";
public static final int DBVERSION = 1;
public static final String TB_TRACKER = "tracker";
public static final String COL_TRACKER_ID = BaseColumns._ID;
public static final String COL_TRACKER_USER = "tracker_user";
public static final String COL_TRACKER_TIME = "tracker_time";
public static final String COL_TRACKER_LATTITUDE = "tracker_lattitude";
public static final String COL_TRACKER_LONGITUDE = "tracker_longitude";
public static final String TB_LASTENTRY = "lastentry";
public static final String COL_LASTENTRY_ID = "lastentry_id";
public static final String COL_LASTENTRY_TIME = "lastentry_time";
public static final String TRG_UPDATE_LASTENTRY = "trg_update_lastentry";
SQLiteDatabase mDB;
public DBHelper(Context context) {
super(context, DBNAME, null, DBVERSION);
mDB = this.getWritableDatabase();
}
@Override
public void onConfigure(SQLiteDatabase db) {
super.onConfigure(db);
}
@Override
public void onCreate(SQLiteDatabase db) {
// tracker(main) table
String crttrackersql = "CREATE TABLE IF NOT EXISTS " + TB_TRACKER +
"(" +
COL_TRACKER_ID + " INTEGER PRIMARY KEY," +
COL_TRACKER_USER + " TEXT," +
COL_TRACKER_TIME + " INTEGER NOT NULL," +
COL_TRACKER_LATTITUDE + " INTEGER," +
COL_TRACKER_LONGITUDE + " INTEGER" +
")";
db.execSQL(crttrackersql);
// lastentry (max time) table - just has 1 row
String crtlastentrysql = "CREATE TABLE IF NOT EXISTS " + TB_LASTENTRY +
"(" +
COL_LASTENTRY_ID + " INTEGER," +
COL_LASTENTRY_TIME + " INTEGER" +
")";
db.execSQL(crtlastentrysql);
//Adds the initial, to be update entry in the lastentry table
String initlastentry = "INSERT INTO " + TB_LASTENTRY + " VALUES(0,0)";
db.execSQL(initlastentry);
//Define the Trigger equivaent of :-
/*
CREATE TRIGGER trg_update_lastentry
AFTER INSERT ON tracker
BEGIN
UPDATE lastentry
SET
lastentry_id = new._id,
lastentry_time = new.tracker_time
WHERE lastentry_time < new.tracker_time
;
END
*/
String crtlastentryupdate = "CREATE TRIGGER IF NOT EXISTS " + TRG_UPDATE_LASTENTRY +
" AFTER INSERT ON " + TB_TRACKER +
" BEGIN " +
" UPDATE " + TB_LASTENTRY +
" SET " + COL_LASTENTRY_ID + " = new." + COL_TRACKER_ID +
", " + COL_LASTENTRY_TIME + " = new." + COL_TRACKER_TIME +
" WHERE " + COL_LASTENTRY_TIME + " < new." + COL_TRACKER_TIME +
";" +
" END";
db.execSQL(crtlastentryupdate);
}
@Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
//Used to delete/ redefine the entire Database
public void restructureDB() {
String droptracker = "DROP TABLE IF EXISTS " + TB_TRACKER;
mDB.execSQL(droptracker);
String droplastentry = "DROp TABLE IF EXISTS " + TB_LASTENTRY;
mDB.execSQL(droplastentry);
String droplastentryupdate = "DROP TRIGGER If EXISTS " + TRG_UPDATE_LASTENTRY;
mDB.execSQL(droplastentryupdate);
onCreate(mDB);
}
//Insert a tracker table entry using current time
public long insertTracker(String user, long lattitude, long longitude) {
return insertTrackerWithTime(user,lattitude,longitude, System.currentTimeMillis());
}
//Insert a tracker table entry specifying the time
public long insertTrackerWithTime(String user, long lattitude, long longitude, long time) {
ContentValues cv = new ContentValues();
cv.put(COL_TRACKER_TIME, time);
cv.put(COL_TRACKER_USER,user);
cv.put(COL_TRACKER_LATTITUDE,lattitude);
cv.put(COL_TRACKER_LONGITUDE,longitude);
return mDB.insert(TB_TRACKER,null,cv);
}
}
MainActivity.java（测试）：-
public class MainActivity extends AppCompatActivity {
DBHelper mDBHelper; // declare DBHelper
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
// Instantiate mDBHelper
mDBHelper = new DBHelper(this);
// Insert some rows with various times (noting highest time is not last)
mDBHelper.insertTracker("Fred",100,100); // NOW
mDBHelper.insertTrackerWithTime("Bert",110,220,System.currentTimeMillis() - (1000 * 60 * 60 * 24));// Less 1 day
mDBHelper.insertTrackerWithTime("Bert",110,220,System.currentTimeMillis() - (1000 * 60 * 60 * 24 * 5)); // less 5 days
mDBHelper.insertTrackerWithTime("Henry",110,220,System.currentTimeMillis() + (1000 * 60 * 60 * 24)); // tomorrow
mDBHelper.insertTrackerWithTime("Bert",110,220,System.currentTimeMillis() - (1000 * 60 * 60 * 24 * 2)); // less 2 days
mDBHelper.insertTrackerWithTime("Bert",110,220,System.currentTimeMillis() - (1000 * 60 * 60 * 24 * 7)); // less 1 week
// get all the inserted rows
Cursor csr = mDBHelper.getWritableDatabase().query(
DBHelper.TB_TRACKER,
null,
null,
null,
null,
null,
null
);
// Shows the rows in the log
while (csr.moveToNext()) {
Log.d("TRACKER",
"User = " + csr.getString(csr.getColumnIndex(DBHelper.COL_TRACKER_USER)) +
"Time = " + csr.getString(csr.getColumnIndex(DBHelper.COL_TRACKER_TIME))
);
}
// get all the lastentry table rows (1)
csr = mDBHelper.getWritableDatabase().query(
DBHelper.TB_LASTENTRY,
null,
null,
null,
null,
null,
null
);
// log the values
while (csr.moveToNext()) {
Log.d("MAXTIME", "Maximum Time is " + csr.getString(csr.getColumnIndex(DBHelper.COL_LASTENTRY_TIME)) +
" for ID = " + csr.getString(csr.getColumnIndex(DBHelper.COL_LASTENTRY_ID))
);
}
}
}
结果：-
04-19 00:44:17.037 1645-1645/? D/TRACKER: User = FredTime = 1524098657028
User = BertTime = 1524012257031
04-19 00:44:17.041 1645-1645/? D/TRACKER: User = BertTime = 1523666657033
User = HenryTime = 1524185057036
User = BertTime = 1523925857039
User = BertTime = 1523493857041
04-19 00:44:17.041 1645-1645/? D/MAXTIME: Maximum Time is 1524185057036 for ID = 4
即最大时间是 1524185057036，它反映了 6 行中的第 4 行。
我有一个查询从 table 获取 MAX(time)，GPS 位置包含 300,000 个条目。我有时间索引以及 USER,TIME 索引。
我访问加载器中的位置，每次将新位置插入 table 时都会收到通知，并将刷新加载器信息，从而调用查询
但我的问题是，几乎每次发生这种情况时，UI 都会锁定，直到查询完成。
我知道这个是因为我在提供者中添加了一个Log.d来显示每次执行查询需要多少毫秒。有时查询将在 30 毫秒（ish）内执行，但大多数情况下需要 2000-4000 毫秒，有时甚至高达 6000 毫秒。在此期间 UI 将停止响应。
我有三个问题，
- 加载器的使用不是应该以这种方式解耦 UI 的减速，这样长查询就不会导致 UI 在执行时锁定吗？
- 是否有任何事情会导致查询等待 "lock" 而导致它确实延迟执行，从而使查询花费更长的时间？
- 为什么查询的执行速度似乎有时变化很快，但大多数时候却很慢。
我已尝试将提供程序中的查询简化为以下内容，仅用于测试：
queryBuilder = new SQLiteQueryBuilder();
tables = PositionEntry.TABLE_NAME;
queryBuilder.setTables(tables);
projectionMap = new HashMap();
projectionMap.put(PositionEntry.COLUMN_USER, PositionEntry.COLUMN_USER);
projectionMap.put(PositionEntry.COLUMN_TIME,"MAX("+PositionEntry.COLUMN_TIME + ") AS " +PositionEntry.COLUMN_TIME );
queryBuilder.setProjectionMap(projectionMap);
long start1 = System.currentTimeMillis();
retCursor = queryBuilder.query(mOpenHelper.getReadableDatabase(), null, null, null, PositionEntry.COLUMN_USER, null, null, null);
Log.d(LOG_TAG,"Sub Query took " + (System.currentTimeMillis() - start1) + "ms to complete Rows:" + retCursor.getCount());
我返回了 3 行，这是正确的，并且我根据每个用户的上次时间获得了他们的最后位置。但是查询每次执行需要2秒左右。
//编辑：修正了一个代码拼写错误
也许考虑有一个单行table，当一行被插入主table.
时，最大时间通过触发器自动更新假设一个名为 lastentry 的 table 具有列 lastentry_id（INTEGER 但不是 rowid 的别名）和 lastentry_time（最大时间为整数）并且 master/main table 被称为 tracker id 列和 time 列（加上其他）
然后是下面
CREATE TRIGGER trg_update_lastentry
AFTER INSERT ON tracker
BEGIN
UPDATE lastentry
SET
lastentry_id = new._id,
lastentry_time = new.tracker_time
WHERE lastentry_time < new.tracker_time
;
END
将更新 lastentry table 中的单行（即更改 lastentry_time 列和 lastentry_id 列（参考相应 tracker 的 ID table行)) 如果trackertable中的时间大于lastnetry[=44中的时间=] table.
因此您可以提取最大时间而无需查询 300,000 行主（跟踪器）table。
Testing/Example
DBHelper.java :-
public class DBHelper extends SQLiteOpenHelper {
public static final String DBNAME = "traking";
public static final int DBVERSION = 1;
public static final String TB_TRACKER = "tracker";
public static final String COL_TRACKER_ID = BaseColumns._ID;
public static final String COL_TRACKER_USER = "tracker_user";
public static final String COL_TRACKER_TIME = "tracker_time";
public static final String COL_TRACKER_LATTITUDE = "tracker_lattitude";
public static final String COL_TRACKER_LONGITUDE = "tracker_longitude";
public static final String TB_LASTENTRY = "lastentry";
public static final String COL_LASTENTRY_ID = "lastentry_id";
public static final String COL_LASTENTRY_TIME = "lastentry_time";
public static final String TRG_UPDATE_LASTENTRY = "trg_update_lastentry";
SQLiteDatabase mDB;
public DBHelper(Context context) {
super(context, DBNAME, null, DBVERSION);
mDB = this.getWritableDatabase();
}
@Override
public void onConfigure(SQLiteDatabase db) {
super.onConfigure(db);
}
@Override
public void onCreate(SQLiteDatabase db) {
// tracker(main) table
String crttrackersql = "CREATE TABLE IF NOT EXISTS " + TB_TRACKER +
"(" +
COL_TRACKER_ID + " INTEGER PRIMARY KEY," +
COL_TRACKER_USER + " TEXT," +
COL_TRACKER_TIME + " INTEGER NOT NULL," +
COL_TRACKER_LATTITUDE + " INTEGER," +
COL_TRACKER_LONGITUDE + " INTEGER" +
")";
db.execSQL(crttrackersql);
// lastentry (max time) table - just has 1 row
String crtlastentrysql = "CREATE TABLE IF NOT EXISTS " + TB_LASTENTRY +
"(" +
COL_LASTENTRY_ID + " INTEGER," +
COL_LASTENTRY_TIME + " INTEGER" +
")";
db.execSQL(crtlastentrysql);
//Adds the initial, to be update entry in the lastentry table
String initlastentry = "INSERT INTO " + TB_LASTENTRY + " VALUES(0,0)";
db.execSQL(initlastentry);
//Define the Trigger equivaent of :-
/*
CREATE TRIGGER trg_update_lastentry
AFTER INSERT ON tracker
BEGIN
UPDATE lastentry
SET
lastentry_id = new._id,
lastentry_time = new.tracker_time
WHERE lastentry_time < new.tracker_time
;
END
*/
String crtlastentryupdate = "CREATE TRIGGER IF NOT EXISTS " + TRG_UPDATE_LASTENTRY +
" AFTER INSERT ON " + TB_TRACKER +
" BEGIN " +
" UPDATE " + TB_LASTENTRY +
" SET " + COL_LASTENTRY_ID + " = new." + COL_TRACKER_ID +
", " + COL_LASTENTRY_TIME + " = new." + COL_TRACKER_TIME +
" WHERE " + COL_LASTENTRY_TIME + " < new." + COL_TRACKER_TIME +
";" +
" END";
db.execSQL(crtlastentryupdate);
}
@Override
public void onUpgrade(SQLiteDatabase db, int oldVersion, int newVersion) {
}
//Used to delete/ redefine the entire Database
public void restructureDB() {
String droptracker = "DROP TABLE IF EXISTS " + TB_TRACKER;
mDB.execSQL(droptracker);
String droplastentry = "DROp TABLE IF EXISTS " + TB_LASTENTRY;
mDB.execSQL(droplastentry);
String droplastentryupdate = "DROP TRIGGER If EXISTS " + TRG_UPDATE_LASTENTRY;
mDB.execSQL(droplastentryupdate);
onCreate(mDB);
}
//Insert a tracker table entry using current time
public long insertTracker(String user, long lattitude, long longitude) {
return insertTrackerWithTime(user,lattitude,longitude, System.currentTimeMillis());
}
//Insert a tracker table entry specifying the time
public long insertTrackerWithTime(String user, long lattitude, long longitude, long time) {
ContentValues cv = new ContentValues();
cv.put(COL_TRACKER_TIME, time);
cv.put(COL_TRACKER_USER,user);
cv.put(COL_TRACKER_LATTITUDE,lattitude);
cv.put(COL_TRACKER_LONGITUDE,longitude);
return mDB.insert(TB_TRACKER,null,cv);
}
}
MainActivity.java（测试）：-
public class MainActivity extends AppCompatActivity {
DBHelper mDBHelper; // declare DBHelper
@Override
protected void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
// Instantiate mDBHelper
mDBHelper = new DBHelper(this);
// Insert some rows with various times (noting highest time is not last)
mDBHelper.insertTracker("Fred",100,100); // NOW
mDBHelper.insertTrackerWithTime("Bert",110,220,System.currentTimeMillis() - (1000 * 60 * 60 * 24));// Less 1 day
mDBHelper.insertTrackerWithTime("Bert",110,220,System.currentTimeMillis() - (1000 * 60 * 60 * 24 * 5)); // less 5 days
mDBHelper.insertTrackerWithTime("Henry",110,220,System.currentTimeMillis() + (1000 * 60 * 60 * 24)); // tomorrow
mDBHelper.insertTrackerWithTime("Bert",110,220,System.currentTimeMillis() - (1000 * 60 * 60 * 24 * 2)); // less 2 days
mDBHelper.insertTrackerWithTime("Bert",110,220,System.currentTimeMillis() - (1000 * 60 * 60 * 24 * 7)); // less 1 week
// get all the inserted rows
Cursor csr = mDBHelper.getWritableDatabase().query(
DBHelper.TB_TRACKER,
null,
null,
null,
null,
null,
null
);
// Shows the rows in the log
while (csr.moveToNext()) {
Log.d("TRACKER",
"User = " + csr.getString(csr.getColumnIndex(DBHelper.COL_TRACKER_USER)) +
"Time = " + csr.getString(csr.getColumnIndex(DBHelper.COL_TRACKER_TIME))
);
}
// get all the lastentry table rows (1)
csr = mDBHelper.getWritableDatabase().query(
DBHelper.TB_LASTENTRY,
null,
null,
null,
null,
null,
null
);
// log the values
while (csr.moveToNext()) {
Log.d("MAXTIME", "Maximum Time is " + csr.getString(csr.getColumnIndex(DBHelper.COL_LASTENTRY_TIME)) +
" for ID = " + csr.getString(csr.getColumnIndex(DBHelper.COL_LASTENTRY_ID))
);
}
}
}
结果：-
04-19 00:44:17.037 1645-1645/? D/TRACKER: User = FredTime = 1524098657028
User = BertTime = 1524012257031
04-19 00:44:17.041 1645-1645/? D/TRACKER: User = BertTime = 1523666657033
User = HenryTime = 1524185057036
User = BertTime = 1523925857039
User = BertTime = 1523493857041
04-19 00:44:17.041 1645-1645/? D/MAXTIME: Maximum Time is 1524185057036 for ID = 4
即最大时间是 1524185057036，它反映了 6 行中的第 4 行。