Comments (4)
You can use this class as a drop-in solution to index pdf files - it uses PdfSharp and PdfSharpTextExtractor
public class PdfContentSearchExtension : ISearchDocumentBuilderExtension
{
public void Populate(SearchDocumentBuilder searchDocumentBuilder, IData data)
{
if (!(data is IMediaFile mediaFile))
{
return;
}
if (searchDocumentBuilder.TextParts.Any() && !String.IsNullOrEmpty(searchDocumentBuilder.Url))
{
return;
}
var mimeType = MimeTypeInfo.GetCanonical(mediaFile.MimeType);
if (!IsIndexableMimeType(mimeType))
{
return;
}
var text = GetText(mediaFile);
if (String.IsNullOrWhiteSpace(text))
{
return;
}
searchDocumentBuilder.TextParts.Add(text);
searchDocumentBuilder.Url = MediaUrls.BuildUrl(mediaFile, UrlKind.Internal);
Log.LogInformation("PdfContentSearchExtension", $"{mediaFile.FileName} indexed successfully");
}
private static string GetText(IMediaFile mediaFile)
{
var sb = new StringBuilder();
using (var pdfDocument = PdfReader.Open(mediaFile.GetReadStream(), PdfDocumentOpenMode.ReadOnly))
{
var extractor = new Extractor(pdfDocument);
foreach (var page in pdfDocument.Pages)
{
extractor.ExtractText(page, sb);
sb.AppendLine();
}
}
return sb.ToString();
}
private static bool IsIndexableMimeType(string mimeType)
{
return mimeType == "application/pdf";
}
}
Just register it in your startup handler like this
public static void ConfigureServices(IServiceCollection serviceCollection)
{
serviceCollection.AddSingleton<ISearchDocumentBuilderExtension>(new PdfContentSearchExtension());
Log.LogInformation("Searching", "PdfContentSearchExtension registered");
}
from c1-cms-foundation.
Fantastic... Thanks, it works!
It sounds like Orckestra.Search.MediaContentIndexing cannot be used at all on Azure. Is that correct?
from c1-cms-foundation.
Since all the indexing of MediaContentIndexing relies on using the IFilter interface it must be safe to assume that it can't do any indexing on Azure.
The above code was even made for a regular Windows server - i believe IFilter has been depricated for many years - not only on Azure.
That leaves us with docx and other types of non-pdfs not being indexed and searchable without writing custom code for that too.
from c1-cms-foundation.
It should be fairly easy though to replace PdfSharpTextExtractor with TikaOnDotnet.TextExtractor which is a library that on paper supports a various of formats
https://kevm.github.io/tikaondotnet/
from c1-cms-foundation.
Related Issues (20)
- Possibility of grouping FunctionParameters i Razor functions
- Composite NuGet package CompositeC1.Core vs C1CMS.Assemblies HOT 5
- Bug when deleting parent record, CascadeDelete is set to false and was supposed to clear the reference to null
- Linking to Media Items in C1 CMS v6.11 HOT 3
- Items can't be duplicated with 8-char random string ID based data types
- Upgrading to C1 6.12 HOT 7
- ImageResizer not working if not logged in (after upgrade to 6.12) HOT 1
- "Please Upgrade - To Stay Safe" - Question about notification HOT 1
- DataTypeDescriptor Clone does not clone "Cachable"
- CompositeC1.Core 6.12 in NuGet HOT 1
- Extend IMediaFile
- Upgrading to C1 6.13 HOT 1
- Cannot use Composite.Core.Routing.Pages.C1PageRoute.RegisterPathInfoUsage() on secondary hostnames' home page
- Allow for various extensionpoint-classes to be DI-friendly
- Expired SSL Certificate on https://c1console.composite.net/C1.aspx HOT 4
- Donut caching breaks functions using C1PageRoute.RegisterPathInfoUsage();
- Weird behavior: Prevent RazorTemplate from executing functions when binding placeholders HOT 2
- Scale images in mediabrowser and mediapicker
- Show WebP files in the Image selector dialog
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from c1-cms-foundation.