Menu

How to Detect and Remove Duplicate Records in SQL Server

Duplicate records in SQL Server can lead to inaccurate reporting, data inconsistencies, and performance issues. In this article, we’ll go over how to identify and safely remove duplicate rows while keeping at least one unique record.

Detecting Duplicates

To find duplicate records in a table, use the GROUP BY and HAVING clauses to count occurrences of each unique combination of values:

SELECT column1, column2, COUNT(*)
FROM YourTable
GROUP BY column1, column2
HAVING COUNT(*) > 1;

Replace column1, column2 with the columns that define a duplicate in your dataset.

If you need to see the actual duplicate rows, use a ROW_NUMBER() approach:

SELECT *
FROM (
    SELECT *, ROW_NUMBER() OVER (PARTITION BY column1, column2 ORDER BY id) AS row_num
    FROM YourTable
) t
WHERE row_num > 1;

Here, id should be a unique column to order the duplicates.

Removing Duplicates

Method 1: Using ROW_NUMBER()

The safest way to remove duplicates while keeping one unique record is by using ROW_NUMBER().

WITH CTE AS (
    SELECT *, ROW_NUMBER() OVER (PARTITION BY column1, column2 ORDER BY id) AS row_num
    FROM YourTable
)
DELETE FROM CTE WHERE row_num > 1;

This deletes all duplicate records while keeping the first occurrence.

Method 2: Using DISTINCT INTO a New Table

If you want to be extra cautious, create a new table with only unique records:

SELECT DISTINCT * INTO NewTable FROM YourTable;

Then, drop the old table and rename NewTable back to YourTable.

Final Thoughts

Always backup your data before running delete operations to prevent accidental data loss. By regularly cleaning up duplicates, you can keep your SQL Server database efficient and reliable.

1
23

Related

Slow initial load times can drive users away from your React application. One powerful technique to improve performance is lazy loading - loading components only when they're needed.

Let's explore how to implement this in React.

The Problem with Eager Loading

By default, React bundles all your components together, forcing users to download everything upfront. This makes navigation much quicker and more streamlined once this initial download is complete.

However, depending on the size of your application, it could also create a long initial load time.

import HeavyComponent from './HeavyComponent';
import AnotherHeavyComponent from './AnotherHeavyComponent';

function App() {
  return (
    <div>
      {/* These components load even if user never sees them */}
      <HeavyComponent />
      <AnotherHeavyComponent />
    </div>
  );
}

React.lazy() to the Rescue

React.lazy() lets you defer loading components until they're actually needed:

import React, { lazy, Suspense } from 'react';

// Components are now loaded only when rendered
const HeavyComponent = lazy(() => import('./HeavyComponent'));
const AnotherHeavyComponent = lazy(() => import('./AnotherHeavyComponent'));

function App() {
  return (
    <div>
      <Suspense fallback={<div>Loading...</div>}>
        <HeavyComponent />
        <AnotherHeavyComponent />
      </Suspense>
    </div>
  );
}

Route-Based Lazy Loading

Combine with React Router for even better performance:

import React, { lazy, Suspense } from 'react';
import { BrowserRouter, Routes, Route } from 'react-router-dom';

const Home = lazy(() => import('./pages/Home'));
const Dashboard = lazy(() => import('./pages/Dashboard'));
const Settings = lazy(() => import('./pages/Settings'));

function App() {
  return (
    <BrowserRouter>
      <Suspense fallback={<div>Loading...</div>}>
        <Routes>
          <Route path="/" element={<Home />} />
          <Route path="/dashboard" element={<Dashboard />} />
          <Route path="/settings" element={<Settings />} />
        </Routes>
      </Suspense>
    </BrowserRouter>
  );
}

Implement these techniques in your React application today and watch your load times improve dramatically!

0
73

Raw string literals in C# provide a flexible way to work with multiline strings, with some interesting rules around how quotes work.

The key insight is that you can use any number of double quotes (three or more) to delimit your string, as long as the opening and closing sequences have the same number of quotes.

The Basic Rules

  1. You must use at least three double quotes (""") to start and end a raw string literal
  2. The opening and closing quotes must have the same count
  3. The closing quotes must be on their own line for proper indentation
  4. If your string content contains a sequence of double quotes, you need to use more quotes in your delimiter than the longest sequence in your content

Examples with Different Quote Counts

// Three quotes - most common usage
string basic = """
    This is a basic
    multiline string
    """;

// Four quotes - when your content has three quotes
string withThreeQuotes = """"
    Here's some text with """quoted""" content
    """";

// Five quotes - when your content has four quotes
string withFourQuotes = """""
    Here's text with """"nested"""" quotes
    """"";

// Six quotes - for even more complex scenarios
string withFiveQuotes = """"""
    Look at these """""nested""""" quotes!
    """""";

The N+1 Rule

The general rule is that if your string content contains N consecutive double quotes, you need to wrap the entire string with at least N+1 quotes. This ensures the compiler can properly distinguish between your content and the string's delimiters.

// Example demonstrating the N+1 rule
string example1 = """
    No quotes inside
    """; // 3 quotes is fine

string example2 = """"
    Contains """three quotes"""
    """"; // Needs 4 quotes (3+1)

string example3 = """""
    Has """"four quotes""""
    """""; // Needs 5 quotes (4+1)

Practical Tips

  • Start with three quotes (""") as your default
  • Only increase the quote count when you actually need to embed quote sequences in your content
  • The closing quotes must be on their own line and should line up with the indentation you want
  • Any whitespace to the left of the closing quotes defines the baseline indentation
// Indentation example
string properlyIndented = """
    {
        "property": "value",
        "nested": {
            "deeper": "content"
        }
    }
    """; // This line's position determines the indentation

This flexibility with quote counts makes raw string literals extremely versatile, especially when dealing with content that itself contains quotes, like JSON, XML, or other structured text formats.

1
69

Reading a file line by line is useful when handling large files without loading everything into memory at once.

✅ Best Practice: Use File.ReadLines() which is more memory efficient.

Example

foreach (string line in File.ReadLines("file.txt"))
{
    Console.WriteLine(line);
}

Why use ReadLines()?

Reads one line at a time, reducing overall memory usage. Ideal for large files (e.g., logs, CSVs).

Alternative: Use StreamReader (More Control)

For scenarios where you need custom processing while reading the contents of the file:

using (StreamReader reader = new StreamReader("file.txt"))
{
    string? line;
    while ((line = reader.ReadLine()) != null)
    {
        Console.WriteLine(line);
    }
}

Why use StreamReader?

Lets you handle exceptions, encoding, and buffering. Supports custom processing (e.g., search for a keyword while reading).

When to Use ReadAllLines()? If you need all lines at once, use:

string[] lines = File.ReadAllLines("file.txt");

Caution: Loads the entire file into memory—avoid for large files!

3
224