On the Evolution of Clusters of Near-Duplicate Web Pages

Click here to download now

Source: Microsoft

Date: January 2008

Type: White Paper

Rating:

(0)

Overview: This paper describes a large-scale study on the prevalence and evolution of clusters of very similar (""Near-duplicate"") web pages. It confirms Broder et al.’s observation of widespread duplication of web pages. In particular, the paper found that about 28% of all web pages are duplicates of some pages in the remaining 72%, and 22% are virtually identical. It examines the documents in the 20 largest clusters and categorized them. The present study also examines the rate at which documents exit clusters, and found that clusters are fairly stable over time; clusters of intermediate size are generally the least stable.

DOWNLOAD NOW

Not what you're looking for? Search again

More From the Internet Section

View All Categories

Business Intelligence : Analytics, Business Process Management, Content management, Dashboards, Data Mining, Performance Management, Databases, Datamarts/Data Warehouses, Information Management, Knowledge Management, Data Quality

Development : Open Source, Windows/.NET, Web Development, Security, Mobility, Java, High Performance Computing, Embedded Systems, Development Tools, Database, Architecture & Design, C/C++

Government : Cloud/SaaS, Leadership, Information Management, Federal, Mobile & Wireless, State & Local, Enterprise Applications, Security, Policy & Regulation, Enterprise Architecture

Hardware : Virtualization Hardware, Windows Servers, Utility/On-demand Computing, Unix/Linux servers, Supercomputers, Peripherals, Macintosh, Handhelds/PDAs, Grid/Cluster Computing, Desktops/PCs, Data centers, Blades, Processors

Healthcare : Interoperability, Administration systems, Clinical information systems, Electronic medical records, The Patient, Security & Privacy, Leadership, Policy & Regulation, Mobile & Wireless

Infrastructure : ATM, Ethernet/Gigabit Ethernet, Frame relay, IPv6, Traffic Management, Network/Systems Management, PBXs, Printers, Remote Access, Routers, Switches, UPS, VPNs, WAN Optimization/Acceleration, Wide Area File Services

Internet : B2B, B2C, Browsers, E-Business/E-Commerce, E-retail, Google, Social Business, Internet Security, Search, Social Networks, Traffic Reporting/Monitoring, Web 2.0, Web Development, Internet Policy

Management : Career Development, Training, Small-Medium Business, Salary/Compensation, ROI/TCO, Regulation/Compliance, Recruiting, Personnel Management, Outsourcing, Legal, H-1B, Executive Insights/Interviews, Workplace Trends

Mobility : WLAN, Wireless Security, Wi-Fi/WiMax, Wi-Fi VOIP, Smartphones, 3G Wireless/Broadband, Muni Wireless, Mobile Messaging, Mobile Business, Fixed Mobile Convergence, 802.11x, RFID

Personal Tech : Blackberry, Bluetooth, Bluray, Digital Cameras, Digital Music, Digital Rights Management, Virtual worlds, iPhone, iPod, Peripherals, Smartphones, TVs/Home Theater, Global Positioning Systems

Security : Security Administration, End user/Client Security, Encryption, Cyberterror, Attacks/Breaches, Application Security, Antivirus, NAC, Perimeter Security, Privacy, Vulnerabilities and Threats, Storage Security, Intrusion Prevention

Services : Telecom/Voice Services, Business Process Outsourcing, Business Services, Disaster Recovery, Systems Integration, Hosted Storage, Internet/Data Services, Outsourcing, Software as a Service, Hosted Applications

Software : Web Services, Service Oriented Architecture, Server Virtualization, Productivity Applications, Operating Systems, Open Source, Linux, Hosted Software/Applications, ERP, Development Tools, Databases, Database Applications, CRM, Business Systems Management, Integration, Application Optimization

Storage : Data protection, Disaster Recovery, Removable/Portable Storage, Security, Storage Fabrics, Storage Systems, Virtualization

Telecom : VOIP, Unified Communications, Voice services, PBXs, Internet policy, Presence, Collaboration Systems, Business, Regulation, Call Centers

Windows/Microsoft : Applications, Internet Explorer, Microsoft Company News, Security, Open Source, Operating system, Office Suite

More Internet Resources

High Bandwidth Internet Access: Opening Doors to New Capabilities
High bandwidth, and the content that rides over it, enables rapid acceleration of business development and the type of collaboration that drives innovation. Companies are ......

Capturing The Potential Of Cloud - How Cloud Drives Value In IT Enterprise
Read this paper to better understand how cloud computing is helping enterprises, governments and industries around the globe meet their current challenges through key value drivers.

More On Internet

Blogs