SQL Server 刪除重復(fù)記錄的幾種方法

字號:


    例如:
    id name value
    1 a pp
    2 a pp
    3 b iii
    4 b pp
    5 b pp
    6 c pp
    7 c pp
    8 c iii
    id是主鍵
    要求得到這樣的結(jié)果
    id name value
    1 a pp
    3 b iii
    4 b pp
    6 c pp
    8 c iii
    方法1
    delete YourTable
    where [id] not in (
    select max([id]) from YourTable
    group by (name + value))
    方法2
    delete a
    from 表 a left join(
    select id=min(id) from 表 group by name,value
    )b on a.id=b.id
    where b.id is null
    查詢及刪除重復(fù)記錄的SQL語句
    查詢及刪除重復(fù)記錄的SQL語句
    1、查找表中多余的重復(fù)記錄,重復(fù)記錄是根據(jù)單個字段(peopleId)來判斷
    select * from people
    where peopleId in (select peopleId from people group by peopleId having count(peopleId) > 1)
    2、刪除表中多余的重復(fù)記錄,重復(fù)記錄是根據(jù)單個字段(peopleId)來判斷,只留有rowid最小的記錄
    delete from people
    where peopleId in (select peopleId from people group by peopleId having count(peopleId) > 1)
    and rowid not in (select min(rowid) from people group by peopleId having count(peopleId )>1)
    3、查找表中多余的重復(fù)記錄(多個字段)
    select * from vitae a
    where (a.peopleId,a.seq) in (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)
    4、刪除表中多余的重復(fù)記錄(多個字段),只留有rowid最小的記錄
    delete from vitae a
    where (a.peopleId,a.seq) in (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)
    and rowid not in (select min(rowid) from vitae group by peopleId,seq having count(*)>1)
    5、查找表中多余的重復(fù)記錄(多個字段),不包含rowid最小的記錄
    select * from vitae a
    where (a.peopleId,a.seq) in (select peopleId,seq from vitae group by peopleId,seq having count(*) > 1)
    and rowid not in (select min(rowid) from vitae group by peopleId,seq having count(*)>1)
    (二)
    比方說
    在A表中存在一個字段“name”,
    而且不同記錄之間的“name”值有可能會相同,
    現(xiàn)在就是需要查詢出在該表中的各記錄之間,“name”值存在重復(fù)的項;
    Select Name,Count(*) From A Group By Name Having Count(*) > 1
    如果還查性別也相同大則如下:
    Select Name,sex,Count(*) From A Group By Name,sex Having Count(*) > 1
    (三)
    方法一
    declare @max integer,@id integer
    declare cur_rows cursor local for select 主字段,count(*) from 表名 group by 主字段 having count(*) >; 1
    open cur_rows
    fetch cur_rows into @id,@max
    while @@fetch_status=0
    begin
    select @max = @max -1
    set rowcount @max
    delete from 表名 where 主字段 = @id
    fetch cur_rows into @id,@max
    end
    close cur_rows
    set rowcount 0 方法二
    "重復(fù)記錄"有兩個意義上的重復(fù)記錄,一是完全重復(fù)的記錄,也即所有字段均重復(fù)的記錄,二是部分關(guān)鍵字段重復(fù)的記錄,比如Name字段重復(fù),而其他字段不一定重復(fù)或都重復(fù)可以忽略。
    1、對于第一種重復(fù),比較容易解決,使用
    select distinct * from tableName
    就可以得到無重復(fù)記錄的結(jié)果集。
    如果該表需要刪除重復(fù)的記錄(重復(fù)記錄保留1條),可以按以下方法刪除
    select distinct * into #Tmp from tableName
    drop table tableName
    select * into tableName from #Tmp
    drop table #Tmp
    發(fā)生這種重復(fù)的原因是表設(shè)計不周產(chǎn)生的,增加唯一索引列即可解決。
    2、這類重復(fù)問題通常要求保留重復(fù)記錄中的第一條記錄,操作方法如下
    假設(shè)有重復(fù)的字段為Name,Address,要求得到這兩個字段唯一的結(jié)果集
    select identity(int,1,1) as autoID, * into #Tmp from tableName
    select min(autoID) as autoID into #Tmp2 from #Tmp group by Name,autoID
    select * from #Tmp where autoID in(select autoID from #tmp2)
    最后一個select即得到了Name,Address不重復(fù)的結(jié)果集(但多了一個autoID字段,實際寫時可以寫在select子句中省去此列)
    (四)
    查詢重復(fù)
    select * from tablename where id in (
    select id from tablename
    group by id
    having count(id) > 1
    )
    學(xué)習(xí)sql有一段時間了,發(fā)現(xiàn)在我建了一個用來測試的表(沒有建索引)中出現(xiàn)了許多的重復(fù)記錄。后來總結(jié)了一些刪除重復(fù)記錄的方法,在Oracle中,可以通過唯一rowid實現(xiàn)刪除重復(fù)記錄;還可以建臨時表來實現(xiàn)...這個只提到其中的幾種簡單實用的方法,希望可以和大家分享(以表employee為例)。
    SQL> desc employee
    Name Null? Type
    ----------------------------------------- -------- ------------------
    emp_id NUMBER(10)
    emp_name VARCHAR2(20)
    salary NUMBER(10,2)
    可以通過下面的語句查詢重復(fù)的記錄:
    SQL> select * from employee;
    EMP_ID EMP_NAME SALARY
    ---------- ---------------------------------------- ----------
    1 sunshine 10000
    1 sunshine 10000
    2 semon 20000
    2 semon 20000
    3 xyz 30000
    2 semon 20000
    SQL> select distinct * from employee;
    EMP_ID EMP_NAME SALARY
    ---------- ---------------------------------------- ----------
    1 sunshine 10000
    2 semon 20000
    3 xyz 30000
    SQL> select * from employee group by emp_id,emp_name,salary having count (*)>1
    EMP_ID EMP_NAME SALARY
    ---------- ---------------------------------------- ----------
    1 sunshine 10000
    2 semon 20000
    SQL> select * from employee e1
    where rowid in (select max(rowid) from employe e2
    where e1.emp_id=e2.emp_id and
    e1.emp_name=e2.emp_name and e1.salary=e2.salary);
    EMP_ID EMP_NAME SALARY
    ---------- ---------------------------------------- ----------
    1 sunshine 10000
    3 xyz 30000
    2 semon 20000
    2. 刪除的幾種方法:
    (1)通過建立臨時表來實現(xiàn)
    SQL>create table temp_emp as (select distinct * from employee)
    SQL> truncate table employee; (清空employee表的數(shù)據(jù))
    SQL> insert into employee select * from temp_emp; (再將臨時表里的內(nèi)容插回來)
    ( 2)通過唯一rowid實現(xiàn)刪除重復(fù)記錄.在Oracle中,每一條記錄都有一個rowid,rowid在整個數(shù)據(jù)庫中是唯一的,rowid確定了每條記錄是在Oracle中的哪一個數(shù)據(jù)文件、塊、行上。在重復(fù)的記錄中,可能所有列的內(nèi)容都相同,但rowid不會相同,所以只要確定出重復(fù)記錄中那些具有最大或最小rowid的就可以了,其余全部刪除。
    SQL>delete from employee e2 where rowid not in (
    select max(e1.rowid) from employee e1 where
    e1.emp_id=e2.emp_id and e1.emp_name=e2.emp_name and e1.salary=e2.salary);--這里用min(rowid)也可以。
    SQL>delete from employee e2 where rowid <(
    select max(e1.rowid) from employee e1 where
    e1.emp_id=e2.emp_id and e1.emp_name=e2.emp_name and
    e1.salary=e2.salary);
    (3)也是通過rowid,但效率更高。
    SQL>delete from employee where rowid not in (
    select max(t1.rowid) from employee t1 group by
    t1.emp_id,t1.emp_name,t1.salary);--這里用min(rowid)也可以。
    EMP_ID EMP_NAME SALARY
    1 sunshine 10000
    3 xyz 30000
    2 semon 20000
    SQL> desc employee
    Name Null? Type
    ----------------------------------------- -------- ------------------
    emp_id NUMBER(10)
    emp_name VARCHAR2(20)
    salary NUMBER(10,2)
    可以通過下面的語句查詢重復(fù)的記錄:
    SQL> select * from employee;
    EMP_ID EMP_NAME SALARY
    ---------- ---------------------------------------- ----------
    1 sunshine 10000
    1 sunshine 10000
    2 semon 20000
    2 semon 20000
    3 xyz 30000
    2 semon 20000
    SQL> select distinct * from employee;
    EMP_ID EMP_NAME SALARY
    ---------- ---------------------------------------- ----------
    1 sunshine 10000
    2 semon 20000
    3 xyz 30000
    SQL> select * from employee group by emp_id,emp_name,salary having count (*)>1
    EMP_ID EMP_NAME SALARY
    ---------- ---------------------------------------- ----------
    1 sunshine 10000
    2 semon 20000
    SQL> select * from employee e1
    where rowid in (select max(rowid) from employe e2
    where e1.emp_id=e2.emp_id and
    e1.emp_name=e2.emp_name and e1.salary=e2.salary);
    EMP_ID EMP_NAME SALARY
    ---------- ---------------------------------------- ----------
    1 sunshine 10000
    3 xyz 30000
    2 semon 20000
    2. 刪除的幾種方法:
    (1)通過建立臨時表來實現(xiàn)
    SQL>create table temp_emp as (select distinct * from employee)
    SQL> truncate table employee; (清空employee表的數(shù)據(jù))
    SQL> insert into employee select * from temp_emp; (再將臨時表里的內(nèi)容插回來)
    ( 2)通過唯一rowid實現(xiàn)刪除重復(fù)記錄.在Oracle中,每一條記錄都有一個rowid,rowid在整個數(shù)據(jù)庫中是唯一的,rowid確定了每條記錄是在Oracle中的哪一個數(shù)據(jù)文件、塊、行上。在重復(fù)的記錄中,可能所有列的內(nèi)容都相同,但rowid不會相同,所以只要確定出重復(fù)記錄中那些具有最大或最小rowid的就可以了,其余全部刪除。
    SQL>delete from employee e2 where rowid not in (
    select max(e1.rowid) from employee e1 where
    e1.emp_id=e2.emp_id and e1.emp_name=e2.emp_name and e1.salary=e2.salary);--這里用min(rowid)也可以。
    SQL>delete from employee e2 where rowid <(
    select max(e1.rowid) from employee e1 where
    e1.emp_id=e2.emp_id and e1.emp_name=e2.emp_name and
    e1.salary=e2.salary);
    (3)也是通過rowid,但效率更高。
    SQL>delete from employee where rowid not in (
    select max(t1.rowid) from employee t1 group by
    t1.emp_id,t1.emp_name,t1.salary);--這里用min(rowid)也可以。
    EMP_ID EMP_NAME SALARY
    ---------- ---------------------------------------- ----------
    1 sunshine 10000
    3 xyz 30000
    2 semon 20000