Prevents storing duplicate files in Laravel with Dedupler

Posted on by




The Problem We All Face

How many times have you opened your storage folder only to find multiple copies of the same file? 😫

We’ve all been there:

  • Users upload the same file multiple times
  • Your storage grows uncontrollably
  • Backups take forever
  • Data becomes inconsistent across your application

I faced this exact problem in projects, and that’s why I built Dedupler for Laravel Storage.



What is Dedupler?

Dedupler is a Laravel package that automatically prevents duplicate file storage using SHA-1 hashing. It gives you a beautiful polymorphic API to manage file attachments while ensuring zero duplicates.



How It Works Under the Hood

// Every file gets SHA-1 hash fingerprint
$hash = sha1_file($file);
// Package benefit happens here:
// - If hash exists → return existing file record
// - If not → store file and create new record
// - All models share the same physical file
Enter fullscreen mode

Exit fullscreen mode



🔧 How to use



1. Using Facade to Store Deduplicated files

/** @var \Illuminate\Http\UploadedFile $file */
/** @var \Maxkhim\Dedupler\Models\UniqueFile $uniqueFile */
$uniqueFile = Dedupler::storeFromUploadedFile($file);
// OR
$uniqueFile = Dedupler::storeFromPath($absolutePathToFile);
// OR
$uniqueFile = Dedupler::storeFromContent($content, 'direct_content_file.ext');
Enter fullscreen mode

Exit fullscreen mode



2. Via Trait to Your Model to keep deduplicated files attached to models

<?php
namespace App\Models;
use Illuminate\Database\Eloquent\Model;
use Maxkhim\Dedupler\Traits\Deduplable;
class Post extends Model
{
    use Deduplable;
}

$post = new Post([...]);
Enter fullscreen mode

Exit fullscreen mode 

/** @var \Illuminate\Http\UploadedFile $file */
/** @var \Maxkhim\Dedupler\Models\UniqueFile $uniqueFile */
$uniqueFile = $post->storeUploadedFile($file);
// OR
$uniqueFile = $post->storeLocalFile($absolutePathToFile);
// OR
$uniqueFile = $post->storeContentFile($content, 'direct_content_file.ext');
Enter fullscreen mode

Exit fullscreen mode



File SHA1 can be checked via RESTapi



Enable RESTapi endpoint to check file existence

Why REST API? This endpoint allows you to check if a file already exists in the system by its SHA-1 hash without uploading it again. Useful for frontend checks or integration with other services.

DEDUPLER_REST_ENABLED=true
Enter fullscreen mode

Exit fullscreen mode



Make HTTP Request

GET http://server.name/api/dedupler/v1/files/{sha1_hash}
Enter fullscreen mode

Exit fullscreen mode



Receive file info (or error message)

{
    "success": true,
    "data": {
        "hash": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
        "sha1_hash": "da39a3ee5e6b4b0d3255bfef95601890afd80709",
        "md5_hash": "d41d8cd98f00b204e9800998ecf8427e",
        "exists": false,
        "filename": "da39a3ee5e6b4b0d3255bfef95601890afd80709.pdf",
        "path": "da\/39\/da39a3ee5e6b4b0d3255bfef95601890afd80709.pdf",
        "mime_type": "application\/pdf",
        "size": 102400,
        "size_human": "100 KB",
        "disk": "deduplicated",
        "status": "completed",
        "created_at": "2025-10-22T18:40:41.000000Z",
        "updated_at": "2025-10-22T18:40:41.000000Z",
        "links_count": 94
    }
}
Enter fullscreen mode

Exit fullscreen mode



🚀 Try It Out!

I’d love your feedback:

⭐ Star the repo if you find it useful – it truly inspires me to keep improving!

What’s your experience with duplicated files management in Laravel? Share your stories and solutions!

P.S. This package was approved by Laravel News!



Source link