Remove Duplicates
Paste or drop a file of text and spreadsheets. Geckodupe removes duplicate rows and lines even when formatting differs: Excel, CSV, JSON, logs, code, and lists.
Results
Remove Duplicates
Drop a folder or zip of project files. Geckodupe finds copies that match even when names and paths differ: identical files, duplicate lines inside documents, and redundant exports.
Results
Remove Duplicate Media
Drop a folder or zip of images and videos. Geckodupe finds files that look the same even when names differ: burst shots, resized copies, re-exports, and near-identical frames.
Results
Despam
Paste form dumps, log lines, or mailing lists. Geckodupe normalizes them so retries and floods become comparable, then removes spam, honeypots, URL floods, disposable addresses, and near-duplicate junk, all in your browser.
API
Create keys and watch usage for hosted normalize, despam, spam prevention, and event idempotency. Sign in on Account first.
Sign in required
API keys and usage live with your account. Create an account or sign back in to continue.
Pricing
Local tools work in the browser with soft caps. Create an account anytime. Basic ($5/mo) unlocks API keys and a daily request allowance. Upgrade when you need production volume. Billing runs through Stripe.
Subscriptions renew monthly until canceled. Change or cancel anytime in the Stripe customer portal. Taxes may apply by location.
Account
Sign in to manage billing, create API keys, and unlock higher local tool limits. Email verification is required before minting keys.
Sign in
Use your password, or switch to a one-time email code.
How It Works
Paste text, upload a file, load a folder or zip, or drop a library of photos and videos. Geckodupe normalizes input so it becomes comparable, then keeps one canonical copy. Select options, click Deduplicate, and get clean output in your browser. No uploads, no servers, and no waiting.
Privacy Guarantee
Everything is processed 100% locally on your device. Zero data ever leaves your browser.
Supported File Types
| Type | Features & Formats |
|---|---|
| Text and Markdown | bullet points and list symbols. |
| Excel (.xlsx) | duplicate rows removed, headers preserved. |
| CSV and TSV | tabular data with header support. |
| JSON, YAML, etc. | canonical key matching. |
| Logs and SQL | strips timestamps, IDs, and serial numbers. |
| Code | Python, JavaScript, TypeScript, HTML, CSS, Shell, and more. |
| Todo and Lists | checkbox state and status prefixes. |
| Images | JPG, PNG, WebP, GIF, BMP, AVIF, TIFF (HEIC if supported). |
| Videos | MP4, MOV, WebM, MKV, AVI, and more. |
Geckodupe
Every tab follows the same idea. Normalize the input so it becomes comparable, fingerprint it, check whether you have already kept something like it, and leave one canonical copy. The biggest value is not removing duplicates. It is that everything first becomes comparable.
Why Deduplicate?
Duplicate rows in spreadsheets, repeated log lines, redundant list items, and near-identical photos all inflate storage and slow down every tool that touches your data. Geckodupe strips the noise so you keep only what matters.
Text / File
Your input is split into lines, trimmed, optionally lowercased, and filtered. Log and SQL modes strip timestamps, UUIDs, and request IDs first so you compare the message, not the metadata. Below 100% similarity, each line becomes a set of word tokens and two lines are close when they share enough words.
J(A, B) = |W(A) ∩ W(B)| / |W(A) ∪ W(B)|
Your similarity slider is the minimum overlap required. Meet it and the line counts as a duplicate. At 100%, only exact keys match after normalization.
duplicate if J(A, B) ≥ θ
Folder / Zip
Geckodupe walks the tree, respects your skip list, and fingerprints each file. Identical byte content produces the same hash, so full copies are grouped and only one path is kept.
h(f) = FNV-1a(contentf)
Each file is routed to the right format engine and line dedupe runs inside it. With cross-file dedupe on, comparison keys are tracked across the whole project. JSON rows are canonicalized with sorted keys first.
exact: keep if k(row) ∉ seen
fuzzy: duplicate if J(k, k′) ≥ θ
Img / Vid
Images decode in your browser, scale to several sizes, and convert to grayscale before hashing so a thumbnail can still match a full export.
Y = 0.299R + 0.587G + 0.114B
Five perceptual hashes capture gradient, brightness, block layout, fine detail, and color placement. Bit mismatches between fingerprints form the Hamming distance.
d(h1, h2) = Σi [h1i ≠ h2i]
Your similarity slider (75% to 100%) sets how many bit differences you allow. The score uses the mean of the two largest hash distances. At 100%, only SHA-256 byte matches count.
match% = 100(1 - d/64)
Simg = 1 - dstrict/64 ≥ 1 - dmax/64
Videos are sampled on a timer. Frames get hashes and hue histograms, timelines are aligned with banded dynamic time warping, and each matched pair is scored.
Sframe = 0.42·Shash + 0.38·Shue + 0.08·Sluma + 0.07·Scontrast + 0.05·Ssat
Burst sequences collapse by comparing filename neighbors. LSH banding and union-find keep large libraries fast.
Folders & zip archives
Load a project or archive, deduplicate files and lines, download a cleaned .zip. Add your own skip paths on the Folder/Zip tab to leave specific files untouched. Spreadsheets inside archives are supported.
Skips node_modules.gitdistvendor
Spam prevention
Normalize and clean form dumps, logs, and lists in the browser. Heuristics catch honeypots, link floods, disposable emails, bait phrases, and near-duplicate junk so retries and floods collapse and you keep only real submissions and lines. For apps and servers, sign in on Account, then open API for a key.
Handles normalizehoneypotsURL floodsAPI keys
Performance & Savings
Duplicate reduction
Average share of repeated lines removed across sample runs. Unique entries stay put as input grows.
Processing speed
lines/sec
Peak throughput at . Largest batch tested: . No uploads, no server queue.
File-size savings
Recovered on a 10k-line sample with heavy duplication ( of lines removed). Log workloads typically drop of noise after timestamp and ID stripping.
Folder / Zip projects
files/sec
Peak scan speed at . The chart shows how scan time grows with file count.
Legal
Terms of Service, Privacy Policy, Acceptable Use, and related notices for Geckodupe, a product of Flareform. By using the website, browser tools, account, billing, documentation, or hosted API, you agree to these terms. Last updated: August 6, 2026.
1. Terms of Service
1.1 Operator and product
Geckodupe is operated by Flareform (“Flareform,” “we,” “us,” or “our”). References to “Service” mean the Geckodupe website, local browser tools (Text Data, Directories, Media, Spam, and related UI), documentation, account system, Stripe billing integration, and hosted API endpoints. “You” means the individual or entity accessing or using the Service.
1.2 Agreement
Accessing or using the Service constitutes acceptance of this Legal page. If you do not agree, do not use the Service. If you use the Service on behalf of an organization, you represent that you have authority to bind that organization, and “you” includes that organization.
1.3 Eligibility
You must be at least 18 years old (or the age of majority where you live) and legally able to enter contracts. Accounts require a valid email address. You are responsible for keeping credentials confidential and for all activity under your account and API keys.
1.4 License to use
Subject to these terms, Flareform grants you a limited, non-exclusive, non-transferable, revocable license to use the Service for your lawful internal or commercial purposes. We retain all right, title, and interest in the Service, branding, documentation, and software. No rights are granted except as expressly stated.
1.5 Your content and local processing
Local browser tools are designed to process data on your device. You are solely responsible for data you paste, upload, select, or process locally, including that you have all rights and permissions needed, and that processing complies with applicable law. Flareform does not claim ownership of your local datasets. You grant Flareform only the limited rights needed to operate hosted features you choose to use (for example, account metadata, API request payloads you send to us, and billing records).
1.6 Changes and availability
We may modify, suspend, or discontinue any part of the Service at any time, with or without notice. We may update these terms by posting a revised Legal page. Continued use after changes constitutes acceptance. We do not guarantee uninterrupted or error-free operation.
1.7 Termination
You may stop using the Service at any time. We may suspend or terminate access, accounts, or API keys if we believe you violated these terms, created risk, failed to pay, or as needed to protect the Service or others. Sections that by nature should survive (including disclaimers, liability limits, indemnity, and governing law) survive termination.
2. Privacy Policy
2.1 Local tools
Text Data, Directories, Media, and local Spam tools are intended to run in your browser. Dataset contents you process with those tools are not uploaded to Flareform for deduplication. Browser storage (for example preferences or session tokens) may remain on your device. Clear site data in your browser to remove local state.
2.2 Account and authentication
If you create an account, we process email address, password hashes or magic-link/session tokens, email verification status, plan and usage metadata, and related security logs. Passwords are stored hashed, not as plaintext. Session tokens authenticate the web app; do not share them.
2.3 Hosted API
When you call hosted endpoints with an API key, we receive and may temporarily process the request payload, headers needed for auth and abuse prevention, timestamps, response metadata, and usage counters. Do not send secrets, regulated health data, payment card numbers, or other sensitive personal data unless you have a lawful basis and accept the risk. You are the controller of data you submit via API; Flareform processes it to provide the Service and enforce limits.
2.4 Billing
Paid plans are processed by Stripe. We receive subscription status, customer identifiers, and related billing metadata. Payment card details are handled by Stripe under Stripe’s terms and privacy policy. We do not store full card numbers on Geckodupe servers.
2.5 Email
We may send transactional email (verification, sign-in codes, password reset, billing notices) through an email provider. Delivery is not guaranteed. Marketing email, if any, will follow applicable consent rules.
2.6 Logs, security, and abuse
We may retain IP addresses, user-agent strings, request paths, error rates, and similar telemetry to operate, secure, rate-limit, and debug the Service. We may block or throttle traffic that appears abusive.
2.7 Cookies and similar tech
The site may use local storage or similar browser storage for sessions and preferences. We do not sell personal information. Third-party CDNs or fonts loaded by the site may receive standard technical requests from your browser.
2.8 Retention and deletion
We retain account, billing, and usage records as needed to provide the Service, meet legal obligations, resolve disputes, and enforce agreements. You may request account closure via the contact channel below; we will delete or anonymize personal account data where reasonably practicable, subject to legal retention needs and backups.
2.9 International users
The Service may be hosted in multiple regions. By using it, you understand your information may be processed where we or our processors operate. Where required by law, you are responsible for ensuring your use is lawful in your jurisdiction.
2.10 Children
The Service is not directed to children under 13 (or under 16 where that is the applicable age). We do not knowingly collect personal information from children.
3. Acceptable Use
You will not, and will not allow others to:
- Violate law, including spam, fraud, harassment, IP infringement, or privacy laws
- Use the Service to send unsolicited bulk messages or evade anti-spam systems without lawful authority
- Probe, scan, or attack infrastructure; bypass rate limits, auth, or billing; or reverse engineer except as allowed by law
- Share API keys publicly, embed keys in client-side apps, or use another user’s credentials
- Upload malware, or process data you are not allowed to process
- Misrepresent affiliation with Flareform or Geckodupe
- Use the Service in a way that risks harm to minors, critical infrastructure, or public safety
We may investigate and take action, including suspension, without prior notice when we reasonably believe a violation occurred.
4. API keys and hosted features
API keys authenticate server-side calls. You are responsible for all usage under your keys, including charges and quota consumption. Rotate or revoke keys if compromised. Hosted results (scores, cleaned text, idempotency decisions) are tools to assist you; you remain responsible for how you act on them in production systems. Open or anonymous API access, if enabled in a deployment, remains subject to these terms and may be revoked.
5. Plans, trials, and billing
Plan names, limits, and prices are described on Pricing and may change. Taxes may apply. Subscriptions renew until canceled through the Stripe customer portal or other method we provide. Fees already charged are generally non-refundable except where required by law or expressly stated by us. Failure to pay may result in downgrade or suspension. Local soft caps for guests and plan quotas are enforcement tools, not a warranty of capacity.
6. Disclaimers
THE SERVICE IS PROVIDED “AS IS” AND “AS AVAILABLE.” To the maximum extent permitted by law, Flareform disclaims all warranties, express or implied, including merchantability, fitness for a particular purpose, title, and non-infringement. We do not warrant that deduplication, spam scores, normalization, media matching, or idempotency will be complete, accurate, or suitable for any particular use case. You are solely responsible for verifying outputs before deleting files, modifying production data, blocking users, or taking automated action. Local tools depend on your browser and device; we are not responsible for browser bugs, lost local data, or interrupted downloads.
7. Limitation of liability; indemnity
7.1 Limitation
To the maximum extent permitted by law, Flareform and its owners, affiliates, contractors, and suppliers will not be liable for any indirect, incidental, special, consequential, exemplary, or punitive damages, or for lost profits, revenue, data, goodwill, or business interruption, arising from or related to the Service or these terms, whether in contract, tort, or otherwise, even if advised of the possibility. Our aggregate liability for all claims relating to the Service will not exceed the greater of (a) the amounts you paid to Flareform for the Service in the three (3) months before the claim arose, or (b) fifty U.S. dollars (US $50). Some jurisdictions do not allow certain limitations; in those cases our liability is limited to the fullest extent permitted.
7.2 Indemnity
You will defend, indemnify, and hold harmless Flareform and its personnel from and against claims, damages, losses, and expenses (including reasonable attorneys’ fees) arising out of your content, your use of the Service, your API traffic, your violation of these terms or law, or your infringement of third-party rights.
7.3 Essential basis
The disclaimers and limits in this Legal page are a fundamental part of the bargain and apply even if any remedy fails of its essential purpose.
8. Intellectual property; feedback
Geckodupe, Flareform, logos, and related marks are trademarks or trade dress of their respective owners. You may not use them without prior written permission except for factual references. Feedback you provide may be used by Flareform without obligation to you.
9. Third-party services
The Service may link to or integrate third parties (including Stripe, email providers, GitHub Pages, CDNs, and Cloudflare). Their terms and privacy policies govern their services. Flareform is not responsible for third-party outages, policies, or acts.
10. Export and sanctions
You may not use the Service if you are prohibited under applicable export control or sanctions laws, or for prohibited end uses.
11. Governing law; disputes
These terms are governed by the laws of the State of Delaware, USA, without regard to conflict-of-law rules, unless mandatory local consumer law requires otherwise. Exclusive venue for disputes that are not resolved informally is the state or federal courts located in Delaware, except that Flareform may seek injunctive relief in any jurisdiction. You waive jury trial to the extent permitted by law. If a provision is unenforceable, the remainder stays in effect. Failure to enforce a provision is not a waiver. These terms are the entire agreement regarding the Service and supersede prior conflicting terms for the same subject.
12. Contact
Questions about this Legal page: use the public repositories and organization contact channels at github.com/orgs/Gekodupe or visit flareform.com. For privacy requests, include enough detail for us to verify and respond.
This page is provided for transparency and risk allocation. It is not personalized legal advice. If your use case requires a signed MSA, DPA, or other contract, contact Flareform before relying on the Service for regulated workloads.
Get Support
Questions about billing, API keys, local tools, or your account. Send a message and we will get back to you.
Contact us
Tell us what you need help with. Do not include API keys or passwords.
Flareform